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ABSTRACT 

In the 1996-1997 school year, the Chicago Public Schools 
(CPS), Illinois, began a national trend when it included a required summer 
program. Summer Bridge, as a central component of its efforts to end social 
promotion. More than 21,000 students in grades 3, 6, and 8, the promotional 
gate grades, have attended Summer Bridge each year,, making it one of the 
largest and most sustained summer programs in the United States. A rigorous 
and careful evaluation as conducted of the Summer Bridge Program using a 
diverse data set that included the analysis of the achievement of all Summer 
Bridge participants, surveys and interviews of participating teachers and 
students, and in-depth classroom observations in 12 schools. Third and sixth 
graders received 90. hours of instruction each summer; eight graders attend 4 
hours a day for 7 weeks for 140 hours. Students are taught by regular CPS 
teachers in small classes, usually at their own schools. CPS requires 
students who score below a given test score on the Iowa Test of Basic Skills 
to attend Summer Bridge. Among the important findings of the evaluation is 
that Summer Bridge has been effective in the short term in producing test 
score gains, especially for sixth and eight graders. The gains produced were 
relatively uniform across demographic and achievement groups. Students were 
extremely positive about their experiences in Summer Bridge. Whether teachers 
knew their students before Summer Bridge was an important predictor of test 
score increases and teacher practice. Higher achieving schools ran more 
effective Summer Bridge programs, and students whose teachers spent more time 
individualizing the curriculum and working with students outside of class had 
greater learning gains than those in other classrooms. Findings show that 
summer programs may be a useful intervention for students who are behind, but 
they are not a substitute effective instruction during the school year. The 
report includes commentaries by Geoffrey D. Borman, Jeffrey C. Valentine, 
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EXECUTIVE SUMMARY 






i n the 1996-97 school year, the Chicago Public Schools (CPS) began a 
national trend when it included a required summer program, Summer 
Bridge, as a central component of its efforts to end social promotion. 
Over 21,000 students in the third, sixth, and eighth, or promotional gate 
grades, have attended Summer Bridge each year, making it one of the larg- 
est and most sustained summer school programs in the country. 

This report presents a rigorous and careful evaluation of Chicago’s Sum- 
mer Bridge program. It is designed to address the following central ques- 
tions that arise in the use of summer programs to support low-achieving 
students under high-stakes testing: 

• To what extent is Summer Bridge effective in increasing stu- 
dents’ test scores and allowing more students to be promoted? 
And, how do the short-term effects of the program vary by 
whether students have very low skills or are closer to the test 
scores required for promotion? 

• To what extent is there evidence that large-scale mandatory 
summer programs can produce uniform effects across students 
and schools? 

• To what extent do summer programs like Summer Bridge pro- 
vide positive learning environments for students? 

• How did staffing characteristics, teachers’ implementation of 
the curriculum, and classroom learning environments shape 
the programs impacts? 

• Can a summer program provide help for low-achieving stu- 
dents that is sustained over time? 
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Over the last four years, a team of researchers at the 
Consortium on Chicago School Research has as- 
sembled a diverse data set to examine these ques- 
tions. We analyzed the achievement of all students 
in Summer Bridge, surveyed and interviewed par- 
ticipating teachers and students about their experi- 
ences, and conducted in-depth classroom observations 
in 12 schools. This report brings together this qualita- 
tive and quantitative research to take a multifaceted 
look at Summer Bridge from its inception through the 
summer of 2000. 

Overview of the Program 

The goal of Summer Bridge is to give low-achieving 
students the extra help they need to remediate popr 
skills and meet test-score cutoffs for promotion. Third 
and sixth graders attend the program three hours per 
day for six weeks for a total of 90 hours of instruction. 
Eighth graders attend four hours per day for seven 
weeks for a total of l40 hours of instruction. Teachers 
are expected to teach a prescribed curriculum that is 
aligned with the Iowa Tests of Basic Skills (ITBS) and 
are provided all classroom materials. Students are 
taught by regular CPS teachers in small classes (ap- 
proximately 16) and most often attend the program 
in their own school. Surveys conducted in 1999 indi- 
cate that eighth-grade Summer Bridge teachers were 
much more likely to have taught in the same grade 
during the previous school year. Reflecting this, over 
twice as many eighth-grade teachers than third- grade 
teachers (4l percent compared with 18 percent) re- 
ported knowing almost all (80 to 100 percent) of their 
Summer Bridge students before the program. 

CPS requires students who score below a given test 
score on the ITBS to attend Summer Bridge. Over 
the course of this study, around one-third of students 
in the promotional grades did not meet these test-score 
cutoffs. Of these students, 97 percent were African- 
American or Latino. Testing rates indicate that par- 
ticipation in Summer Bridge was high. A conservative 
estimate indicates that over 80 percent of the students 
who were required to attend Summer Bridge were re- 
tested at the end of the program. 







Critical Findings 

^ Summer Bridge has been effective in the short 
run in producing test-score gains, particularly 
among sixth and eighth graders, and in allow- 
ing more students to meet the test-score cutoffs 
for promotion. Our results .suggest that sum- 
mer programs may be a promising approach for 
providing students extra instructional time and 
remedial support. Sixth and eighth graders had 
substantial test-score gains. In ail three grades, 
the rate at which Summer Bridge students in- 
creased their test scores was above their rate dur- 
ing the regular school year. 



SI We found little evidence to support one of the 
main concerns about the use of summer pro- 
grams for students under high-stakes testing: 
namely, chat such programs will only produce 
benefits for students who are close to the test- 
score cutoffs. However, one of the most positive 
findings in this report is that Summer Bridge 
produced relatively uniform gains across de- 
mographic and achievement groups. Third 
graders who were at the highest risk of failure 
benefited the most from the program. 

^ Students were extremely positive about their 
experiences in Summer Bridge. Sixth and 
eighth graders portrayed their summer class- 
rooms as environments where they were expected 
to work hard. They also reported that their teach- 
ers were supportive. Most imponandy, we found 
that students who attended Summer Bridge were 
significantly more positive in the summer than 
in the school year about the academic environ- 
ments of their classrooms and the attention they 
received from teachers. 

O Whether teachers knew their students before 
Summer Bridge was an important predictor 
of test-score increases and teacher practice. 
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Ibachers who knew a large proportion of their 
Summer Bridge students beforehand were more 
likely to report adapting the curriculum to meet 
students’ needs and working more closely with 
students outside of class time. These effects were 
most pronounced for older students. I'his sug- 
gests that summer programs may be more effec- 
tive when teachers know their students, are 
familiar with their learning styles and behaviors, 
and are able to extend school year relationships 
into the summer. 

P There is evidence that Summer Bridge students 
had slightly larger learning gains over two years 
than comparable students who did not attend 
the program. Although these results are encour- 
aging, the effects were small. Summer Bridge did 
not substantially alter low- achieving students’ 
school year learning rates; it appeared to help 
keep these students on track, but did not change 
their learning trajectories. The program pro- 
vided a short-term intervention that allowed 
low-achieving students to raise their test 
scores and may have provided an extra boost 
that kept them from falling further behind. 

® Higher achieving schools ran more effective 
Summer Bridge programs. Students who at- 
tended Summer Bridge in schools with higher 
achievement during the school year had larger 
test-score gains than students in lower perform- 
ing schools. These differences were most pro- 
nounced in the third grade. Teachers in higher 
performing schools were more positive about the 
learning environment in Summer Bridge and re- 
ported paying more attention to the individual 
needs of students. These practices were associ- 
ated with larger adjusted test-score increases. 
Thus, we do not find that Summer Bridge, even 
with its mandatory curriculum and uniform 
materials, ameliorated differences in quality 
across schools. 



1^ The Summer Bridge curriculum contributed to 
consistency in topics covered, pacing, and skills 
focus. For the most part, teachers followed the 
curriculum. This seemed to play a critical role 
in ensuring exposure to similar content across 
classrooms. We still found differences in instruc- 
tion across classrooms, however. Our look at in- 
struction showed that the quality of 
interactions between teachers and students 
was a distinguishing factor between the most 
effective Summer Bridge classrooms and the 
average classroom. A relatively small propor- 
tion of teachers taught in ways that engaged stu- 
dents, provided substantive feedback, and 
worked to address individual learning needs. 
Students in these teachers’ classrooms had lars^er 

w 

learning gains in the program. 

S Teacher surveys and classroom observations sug- 
gest that summer programs that rely on manda- 
tory curricula are not “teacher proof” Students 
whose teachers spent more time individual- 
izing the curriculum and working with stu- 
dents outside of class had greater learning 
gains than students in classrooms where 
teachers spent less time adapting the cur- 
riculum and providing individualized at- 
tention. 

We find that summer programs may be a useful 
intervention for students who are behind, but they 
are not a substitute for efiective instruction during 
the school year. Fhere is no evidence that Summer 
Bridge affected school year learning rates nor did it 
address the fact that participating students continued 
to show low performance. Summer Bridge does not 
change students’ experiences during the school year. 
What we have learned is that summer can provide an 
opportunity for teachers to work closely with students 
in an environment that is different from the school 
year and can benefit students who are in need of extra 
support. Indeed, our analysis suggests that part of the 
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reason why students reacted positively to Summer 
Bridge was because it contrasted dramatically with their 
school year experiences. Not surprisingly, when Sum- 
mer Bridge students return to regular school-year en- 
vironments, they appear to return to their previous 
learning rates. Thus, we do hot find evidence that a 
one-time summer intervention is an effective means 
of addressing the long-term learning needs of low- 
achieving students. 

The goal of this report is to inform policy maldng 
both in Chicago and nationally. Even though this re- 
port provides a comprehensive look at Chicago’s Sum- 



mer Bridge program, it is a look at only one program 
embedded in a very specific policy initiative. The in- 
terpretive summary discusses what this study may tell 
us about the effects of summer programs under high- 
stakes testing and what issues might arise both in Chi- 
cago and nationally as policymakers and educators 
grapple with the findings. The report also includes 
commentaries from the following experts on summer 
school and accountability policies: Geoffrey Borman, 
William Clune, John Ponz, and Harris Cooper and 
Jeffrey Valentine. 
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INTRODUCTION AND REPORT OVERVIEW 



ntil recently, the summer offerings of most major school systems were 
an array of small and diverse programs such as remedial or acceler- 
ated summer school classes or programs for special populations. While 
such efforts provided supplemental opportunities, they were not viewed as 
core elements of school systems' instructional programs or reform initia- 
tives. In 1997, the Chicago Public Schools (CPS) began a national trend 
when it made a required summer program, Summer Bridge, a central com- 
ponent of its effort to end social promotion. Mandatory summer pro- 
grams have since expanded rapidly as school administrators struggle with 
how to provide extra support to help students meet the demands of high- 
stakes testing. In the past several years, New York, Los Angeles, Boston, 
San Diego, and Baltimore, among other cities, ran large mandatory sum- 
mer programs.^ Estimates suggest that over a quarter of the nations school 
districts use summer school participation as a criterion for promotion to 
the next grade.” 

Summer programs are both administratively and fiscally costly. They 
require keeping buildings open all year, mounting programs in short peri- 
ods of time, and processing test-score results quickly in order to inform 
students and schools of. their participation. They take away from profes- 
sional development and programming time. And, they require convincing 
tired teachers and students to gear up and commit to working during what 
has historically been seen as sacred vacation time. Is it worth the effort? 
Are mandatory summer programs an effective means of providing support 
to students under high-stakes testing? 

While not the largest, Chicago's Summer Bridge program was certainly 
the harbinger of a national trend, the most ambitious in its design, and 
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the program that was most closely linked to 
Chicago’s successes in ending social promotion. This 
report takes a careful look at Chicago’s experience in 
this effort and the program’s outcomes. Was Summer 
Bridge effective in gaining high rates of participation, 
raising students’ test scores, and increasing their 
chances of promotion.^ How did students view their 
participation in the program and how did teachers ap- 
proach instruction? This report examines the extent 
to which instruction varies across schools and whether 
differences across classrooms in staffing and instruc- 
tion shape what impact the program has. It also looks 
at the longer-term effects of summer programs, a topic 
on which there has been little prior research. 

The Poliqf Context 

The recent expansion ol summer programming in ur- 
ban school systems has largely resulted from the intro- 
duction of high-stakes testing and efforts to end social 
promotion.^ Over the past several years, most major 
school systems in the country have adopted policies 
to end social promotion. Wliile the details of such poli- 
cies vary, a common theme is that students must meet 
minimum test-score cutoffs or be retained. Many 
school systems are struggling, however, with the ques- 
tion of how best to provide low-achieving students the 
extra support they will need to meet these new expec- 
tations. An increasingly popular option is to use the 
summer for extended instructional time. 

Adding extra time is an appealing strategy. Past re- 
search finds that increased instructional time has posi- 
tive effects for low-achieving students.*^ Multiple studies 
also document that impoverished students lose ground 
during the summer months, particularly in reading, 
and that this “summer learning loss” may be an im- 
portant reason why low- income children fall behind 
their more advantaged counterparts.^ All of this sug- 
gests that summer is an opportune time to intervene 
and provide extra support for low-achieving students. 

At the same time, research on the effects of sum- 
mer school has found that low-income students often 
do not benefit as much from summer programs as more 

O 

ERIC 



advantaged students.^ We also know much less about 
the impact of summer support programs under high- 
stakes testing. The mandatory nature of Summer 
Bridge and its highly structured approach may rem- 
edy such problems as low attendance, uneven program 
quality, and short program duration that often plague 
summer programs in urban areas.' 

Critics worry, however, that the strategy of provid- 
ing summer programs under high-stakes testing could 
fall short of the goal of truly improving student achieve- 
ment. First, there is a concern that summer programs 
linked to high-stakes exams will focus primarily on 
short-term test preparation and will not give students 
useful learning experiences that can be sustained over 
time. Second, there is a concern that such efforts will 
not adequately meet the needs of low-achieving stu- 
dents. The short duration of such programs and the 
second chance to pass the test may benefit students 
who are close to the promotional cutoffs, but may leave 
struggling students behind. Third, the size of many of 
these initiatives raises questions about quality. Can large 
school systems have high-quality programming while 
mounting these expansive efforts on top of maintain- 
ing their existing school structure? Without address- 
ing differences in the quality of instruction across 
schools, critics worry that such initiatives will exacer- 
bate existing differences in school quality, with stu- 
dents in better performing schools receiving higher 
quality programming than students in lower perform- 
ing schools. 

This report was written with these policy debates 
in mind. We look at the extent to which program ef- 
fects and student experiences vary by the race, ethnic- 
ity, gender, and achievement characteristics of students 
and of the schools they attend. We also examine the 
extent to which teachers’ perceptions of the effec- 
tiveness of the curriculum and the program vary by 
school composition. 



Summer Bridge in Chicago 

The centerpiece of Chicago’s effort to end social pro- 
motion, which was adopted by the system in the 1996 - 
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97 school year, is a set ot promotional test-score cut- 
offs for third, sixth, and eighth graders. Students in 
these grades must achieve a minimum score on the 
Iowa Tests of Basic Skills (ITBS) in reading and math- 
ematics in order to be promoted to the next grade. 
Students who do not meet the criteria in the spring 
are required to participate in the Summer Bridge pro- 
gram and are retested at the end of the summer. Those 
who again do not meet the criterion are evaluated for 
retention or, if they will be 15 years old by December 
1 of that year, are sent to alternative schools called Aca- 
demic Preparatory Centers. Between 1996 and 2000, 
approximately one-third of third, sixth and eighth 
graders failed to meet the test-score cutoffs by the 
end of the school year. In each year, more than 
21,000 students in these grades were required to 
attend Summer Bridge. 

From the start. Summer Bridge was seen as the in- 
novation that set Chicago's efforts apart from previ- 
ous unsuccessful initiatives to end social promotion 
using high-stakes testing.^Summer Bridge is intended 
to give students the extra help they need to remediate 
poor skills, provide intensive preparation for taking 
the ITBS, and give students a second chance for pro- 
motion. Summer Bridge is distinguished by its small 
class size, highly prescribed and centrally developed 
curriculum, and its mandatory high-stakes approach. 
The average class size in Summer Bridge is 16, There 



is a centrally developed curriculum that is aligned to 
the content on the ITBS. Teachers are provided with 
daily lesson plans and all instructional materials, and 
they are expected to keep up with the pace of the les- 
son plans. Monitors visit classrooms to check if teach- 
ers are maintaining the expected pace.'^ Finally, the 
mandatory nature of the program, with participation 
followed by re- testing, is intended to increase student 
motivation, attendance, and work effort. 

Early evidence on the effects of Summer Bridge 
was promising. In a report on the progress of the 
first group of students who faced the promotional 
standards, researchers at the Consortium on Chicago 
School Research concluded that Summer Bridge and 
the second chance opportunity it afforded was ef- 
fective in raising the proportion of students who 
met the test-score criterion for promotion, This 
early evidence suggested that Summer Bridge might 
be a useful approach to provide support for students 
under Chicago's high-stakes testing policy. At the same 
time, these results also raised a host of new questions. 
Does Summer Bridge actually produce test-score im- 
provements for all low-skilled students, or does it sim- 
ply provide an extra boost for the students closest to 
the cutoffs and leaves the lowest skilled students be- 
hind? Do early positive results mean that Summer 
Bridge provides an effective and supportive learn- 
ing environment, or is the program simply test 



The Ending Social Promotion Study 

This report is one in a series of Consortium on Chicago School Research reports that began in 1999 with the 
publication of Ending Social Promotion: Results from the First Two Years. An update to this report was pub- 
lished in 2000. The first report provided initial findings on students’ passage through the promotional gates; 
passing rates; Iowa Tests of Basic Skills achievement trends; and differences in exclusion, passing, waiver, and 
retention rates. The Consortium will be releasing two more major reports on ending social promotion. The first 
is an analysis of changes in instruction in the Chicago Public Schools (CPS) since the introduction of account- 
ability for students and schools in 1 996. The final report will be an extensive analysis of grade retention in CPS 
since this set of policies was adopted. That report will examine the effects of grade retention on students 
and on the school system as a whole. It will also include analysis of the impact of this initiative on dropout 
rates, students in special education, bilingual education, and on those who are sent to Academic Prepara- 
tory Centers. 
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preparation geared to raising students’ scores on one 
test? Finally, what is the importance of each of the 
programs components — small class size, mandatory 
curricula, and high-stakes approach — in producing 
these results? 

In order to address these questions, we must look 
beyond passing rates to examine test-score increases, 
student experiences, and instruction. Since 1998, a 
team of researchers has assembled a large and diverse 
data set that enables us to explore critical aspects of 
Summer Bridge including student and teacher experi- 
ences, the nature of instruction, and the programs 
short- and long-term effects on test scores. We ana- 
lyzed student achievement in the program, surveyed 
teachers and students in Summer Bridge, interviewed 
teachers and students about their experiences, and 
conducted in-depth classroom observations in 12 
schools. This report brings together this qualitative and 
quantitative research for a multifaceted look at Sum- 
mer Bridge. 

Report Overview 

Chapter 1 begins by providing a basic overview of the 
Summer Bridge student population and the charac- 
teristics of the program. We examine the number of 
students who attended the program from 1 997 to 2000 
and describe their demographic and achievement char- 
acteristics. We then look at the basic characteristics of 
the program. Do most schools staff their Summer 
Bridge classrooms with teachers who had taught there 
previously? Are teachers teaching in the same grades 
and subjects as during the school year? Do these char- 
acteristics vary across grades? Wliat does it mean that 
the Summer Bridge curriculum is aligned to the ITBS? 
Finally, what skills does the program emphasize, and 
how is the curriculum structured? 

Chapter 2 examines the shoa-term effects of Sum- 
mer Bridge on test-score gains and on the proportion 
of students who meet the test-score cutoffs for pro- 
motion. This chapter estimates achievement gains in 
the program in each of the programs first four years 
(1997 through 2000) and by students’ prior levels of 



achievement, demographic characteristics, and the 
characteristics of the schools they attend. We compare 
test-score increases in Summer Bridge to those in other 
summer programs and to the gains that students who 
attend Summer Bridge experience during the school 
year. Finally, Chapter 2 examines this question: To what 
extent do short-term gains over the summer translate 
into the outcome that is most important for students — 
meeting the test-score cutoffs for promotion? 

Chapter 3 looks at students’ assessments of the qual- 
ity of their experiences in Summer Bridge. This chap- 
ter uses student survey data collected both during the 
school year and summer in 1999, as well as qualitative 

The results presented here raise 
critical questions about the 
generalizability of findings and 
the desipi of effective summer 
progm?ns. Throughout the re- 
port, we have ttied to structure 
our analysis so that it can best 
infonn these policy debates. 

interviews with students to explore how they assessed 
the academic orientation of their classrooms and the 
personal support they received from teachers in Sum- 
mer Bridge. We compare those portrayals to students’ 
assessments from the school year. 

The first three chapters garner evidence for whether 
Summer Bridge provides an effective intervention in 
the short term. In Chapter 4, we turn to the teachers’ 
perspectives. Using results from both teacher surveys 
and interviews, we examine how teachers attempted 
to meet students’ needs in Summer Bridge and their 
evaluation of the curriculum and overall learning en- 
vironment. This chapter looks at the extent to which 
teachers in Summer Bridge differed in their instruc- 
tional approach and level of connection to their stu- 
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dents, and whether these characteristics mattered in 
terms of students test-score gains. 

Chapter 5 takes an in-depth look at instruction in 
Summer Bridge through analysis of classroom obser- 
vations. This chapter is intended to examine the link 
between program design and actual implementation. 
Did teachers actually follow the prescribed lesson plans? 
How much did teachers shape the quality of how the 
curriculum was put into practice? Was instruction in 
Summer Bridge only guided by the topics covered on 
the ITBS or did most teachers simply revert to “teach- 
ing to the test?” Answering these questions is critical if 
we are to understand both why students experience 
test-score increases in Summer Bridge and the role of 
programmatic structure in shaping program quality. 

Ultimately, whether programs are effective depends 
upon whether short-term gains are sustained over time. 
The final chapter of this report presents an analysis of 
whether Summer Bridge test-score gains are sustained. 
It compares the two-year learning gains of students 
who attended Summer Bridge who were slightly be- 
low the test-score cutoffs to those of students who did 
not attend the program because they were slightly 
above the cutoffs. The sharp “discontinuity” in the 
likelihood of program participation created by the 
use of test-score cutoffs provides a comparison group 
for students who attended Summer Bridge, allow- 
ing us to explore whether the outcomes of the two 
groups were similar. 

This report focuses on the students who attended 
Summer Bridge in high -stakes testing grades (third, 
sixth, and eighth) from 1997 to 2000. Over the past 
several years, CPS has expanded summer school to 
encompass other groups, including first and second 
graders with low test scores and all third, sixth, and 
eighth graders with below-grade-level test scores. This 
report does not examine the effects of the Summer 
Bridge expansion. 



We also restrict the analysis to students who were 
first-time Summer Bridge participants. As part of the 
CPS promotional policy, students who were retained 
are required to attend Summer Bridge again if they do 
not meet the promotional test-score cutoffs after a sec- 
ond time in that grade. In related repons, we examine 
the progress of these retained students. This report 
restricts analysis to first-time Summer Bridge enroll- 
ees in order to distinguish between the effects of Sum- 
mer Bridge and the effects of retention. 

Finally, we have asked five scholars to comment on 
this study and our findings, including commentary 
on its strengths and shortcomings, its contribution 
to research and policy, and to place it in the context 
of related research and policy issues. These are in- 
cluded at the end of this report. We thank them for 
their thoughtful contributions. 

The Goal of This Report 

1 he goal of this report is to inform policy making both 
in Chicago and nationally. We hope that school dis- 
tricts and policy makers throughout the United States 
will be able to use this report to engage in discussions 
about their own initiatives and the potential role that 
summer programs can play under high-stakes testing. 
While this report provides a comprehensive look at 
Chicago’s Summer Bridge program, it is a look at only 
one program embedded in a very specific policy ini- 
tiative. The results presented here raise critical ques- 
tions about the generalizability of findings and the 
design of effective summer programs. Throughout the 
report, we have tried to structure our analysis so that 
it can best inform these policy debates. The interpre- 
tative summary discusses what we have learned so far 
and what issues might arise both in Chicago and na- 
tionally as policymakers and educators grapple with 
our findings.*^ 
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T he Chicago public schools system has one of the largest mandatory 
summer school programs in the country. From the programs first year 
in 1997 through 2000, over 21,000 third-, sixth-, and eighth-grade 
students were required to attend each year.’ Summer Bridge is targeted at 
students with the lowest skills. The CPS central administration requires 
that schools provide Summer Bridge and permits little flexibility in the 
program s curriculum and design. This chapter looks at multiple data sets 
to address the following questions: Which students actually attend Sum- 
mer Bridge? Who teaches in the program? And, what is the nature of the 
mandatory curriculum? 

Student Characteristics and Testing Rates in Summer Bridge 

Who Is Required to Attend? 

The centerpiece of the systems efforts to end social promotion is a set of 
minimum test-score cutoffs in reading and mathematics on the Iowa Tests 
Basic Skills (ITBS). Students in the third, sixth, and eighth grades need to 
reach these cutoffs or they are required to attend Summer Bridge, The 
policy does not, however, apply stricdy to all students in these grades. 
Students are not included under this policy if they are in special education 
or have been in bilingual education for four years or fewer." (Initially, the 
bilingual education criterion was three years or less. In 1999, it was ex- 
tended to four years.) 

Initially, the CPS promotional test-score cutoffs were relatively low: 1.0 
year below grade level for third graders, 1.5 years below grade level for 
sixth graders, and 1.8 years below grade level for eighth graders. Even with 
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these low test-score cutoffs, a high proportion of in- 
cluded students failed to meet the promotional crite- 
ria and was required to attend Summer Bridge. In 
1997, the first year of the program, almost half of third 
graders (48 percent), 35 percent of sixth graders, and 
27 percent of eighth graders failed to meet the pro- 
motional criteria at the end of the school year and were 
required to attend Summer Bridge (see Figure 1-1). 
The number of third- and sixth-grade students who 
did not meet the cutoff was highest in the first year. 
The number of eighth graders who did not meet the 
cutoff, however, has not declined, as the eighth-grade 
test-score cutoffs have been raised each year. 

Although a high proportion of Chicago students in 
the promotional gate grades were required to attend 
Summer Bridge, the odds of attending depended, to a 
large extent, on students’ prior achievement. We evalu- 
ated the chance that students at different achievement 
levels would have to attend Summer Bridge based on 
how far behind the promotional test-score cutoffs stu- 
dents were upon entry into the promotional gate grade. 
Because there is often variation in a student’s perfor- 
mance from test to test (aside from expected test-score 
increases from year to year), any one test score will 
have some error of measurement. To obtain the best 



estimate of students’ true abilities, we examined stu- 
dents’ entire test score trajectories prior to the promo- 
tional gate grade rather than their last observed test 
score. ^ Because our results did not differ across pro- 
gram years, we present results for 1999. 

We grouped students into three different risk cat- 
egories: high, moderate, and low. Students who would 
have to increase their reading test scores by over 1.5 
grade equivalents (GEs) in one year — over twice the 
normal test-score improvement rate for low-achieving 
students — to make the promotional cutoff were con- 
sidered high risk. Students who needed average to 
above average increases (0.5 to 1.5 GEs) were charac- 
terized as moderate risk. Students who needed increases 
that were less than 0.5 GEs were considered low risk. 

Very few high-risk third and sixth graders met the 
cutoffs for their grade (See Figure 1 -2). Thus, the pro- 
motional policy had the effect of requiring summer 
school attendance for all students with the lowest ITBS 
test scores. Passing rates were significantly higher 
among students at moderate risk. Approximately 64 
percent of third-grade students and less than half of 
sixth- and eighth-grade students whom we character- 
ized as being at moderate risk of retention were re- 
quired to attend Summer Bridge. Finally, very few 



Data Used in This Chapter 

This chapter draws on two primary sources of data. First, this report draws on student test- score files and 
other administrative records provided by CPS. These official school records include demographic infor- 
mation on students (age, gender, race, and ethnicity), test scores on the Iowa Tests of Basic Skills for 
students' entire school careers, and administrative information such as students' grade level, schools 
attended, and special education/bilingual status. 

Second, in summer 1999, the Consortium on Chicago School Research surveyed all Summer Bridge 
teachers. The Summer Bridge surveys, administered during the last week of the program, included ques- 
tions on teachers' backgrounds and classroom composition. The survey asked teachers to assess the 
quality of the Summer Bridge curriculum and materials, to report on their instruction, and to assess the 
needs and performance of their students. Surveys were collected from 1,335 teachers (556 third grade, 
434 sixth grade, and 345 eighth grade). Of the 341 schools that had third graders in the 1999 Summer 
Bridge program, 295, or 87 percent, participated in the survey, A total of 323 schools had sixth-grade 
Summer Bridge in 1 999, and 84 percent participated in the survey. The proportion is slightly smaller in the 
eighth grade, where 297 schools ran eighth-grade Summer Bridge, and 234, or 79 percent, participated. 
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FIGURE M 

More Third Graders Than Sixth Graders Had to 
Attend Summer Bridge 

Number of Third, Sixth, and Eighth Graders Required 
to Attend Summer Bridge, 1997-2000 
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Totals for Promotional Gate Grades 


Total tested* 


70,595 
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Number not meeting cutoff, spring 


25,841 


22,654 


20,935 
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Percent not meeting cutoff, spring 


37% 
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35% 



* Total tested refers to the total number of students tested who were included 
under the policy. 



FIGURE 1-2 

Most Students at High Risk of Retention 




Low risk Moderate risk High risk 



■ Third grade ■ Sixth grade ■ Eighth grade 



Risk categories are based on predicted reading scores from the prfer year. 
Numbers are for ail Included students who failed to meet reading and/or math test- 
score cutoffs in spring and were required to attend Summer Bridge. Similar results 
were observed in other years. In 1999, 52% of 3rd graders, 59% of 6th graders, and 
55% of 8th graders were characterized as being at low risk of retention. The 
percentages for moderate risk are: 42% of 3rd graders, 32% of 6th graders, and 
29% of 8th graders. Finally, 6% of 3rd graders, 9% of 6th graders, and 16% of 8th 
graders were characterized as high risk. 



The Chicago Public Schools' Test-Score Cutoffs 



Under the Chicago Public Schools' (CPS) policy, third, sixth, and 
eighth graders need to meet test-score cutoffs on the Iowa Tests 
of Basic Skills (ITBS) in reading and mathematics in order to be 
promoted. The test-score cutoffs are set using the grade equiva- 
lent (GE) metric. One month corresponds to 0.1 GE; therefore, 
10 months equal one academic year or one GE. The ITBS reports 
results in GEs on national norms where a student is considered 
on grade level if, when taking the test in the eighth month of the 
school year, she obtains a score of that grade plus eight months, 
Thus, a third grader is considered to be reading at grade level on 
national norms if her ITBS score is a 3.8. 

Between 1997 and 2000, the CPS promotional test-score cutoff 
for third graders was set at 2.8 GEs, one year below grade level. 
The sixth-grade cutoff was set at 5.3, which was 1.5 years below 
grade level at 6.8. In 2000, this was raised to 5.5. The eighth- 
grade cutoff was initially set at 7.0, 1.8 years below grade 
level at 8.8. In 1998, the cutoff for eighth grade was raised to 
7,2. In 1999, it was raised to 7.4, and it was raised again, to 
7.7, in 2000, 
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FIGURE 1-3 

Almost All Summer Bridge Students Are African-American or Latino 

Raciaf/Ethnic Composition of Students included under the Policy and Required to 
Attend Summer Bridge 



on average/ Approximately two-thirds of 
Summer Bridge sixth and eighth graders 
and over 80 percent ot Summer Bridge 
third graders were African-American. 
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3rd graders under policy 
All 3rd graders 
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Numbers are averaged over 1997-2000. Bilingual program students and special education students are 
excluded from the policy. In third grade, 11% of African-Americans, 62% of Latinos, and 37% of other 
students were excluded. In sixth and eighth grades, about 17% of African-Americans, 24% of Latinos, 
and 26% of other students were excluded from the poilcy. 



low-risk students failed in spring or had to attend Sum- 
mer Bridge. Thus, it does not appear that many stu- 
dents who entered the promotional gate grade with 
test scores already within range ot the test-score cut- 
offs were required to attend the program* 

Like many large urban school systems, CPS serves 
a predominantly minority student body. Over the 
course of this study, over half of CPS students in the 
promotional gate grades were African-American, and 
approximately one-third were Latino (see Figure 1 -3). 
The decision to exclude students who had been in the 
bilingual program for three years or less meant that a 
large proportion of Latino students were excluded from 
the promotional policy. This was particularly true in 
the third grade. Although Latino students represent 
one-third of CPS third graders, they account for only 
18 percent of third graders included in testing. The 
proportion of Summer Bridge students who were 
Latino was quite similar to the proportion of included 
students who were Latino. African-American students, 
however, are substantially overrepresented in Summer 
Bridge both because they are less likely to be excluded 
from the policy and because, among included students, 
African-Americans have substantially lower test scores 



What Are the Achievefnent 
Charaderisfics of Summer 
Bridge Students? 

Our look at students’ risk of attending 
Summer Bridge suggests that most of the 
students with the lowest skills in the pro- 
motional gate grades did not meet the 
cutoffs in the spring. How far behind was 
the average Summer Bridge student when 
entering? In 1997, the average ITBS read- 
ing score of a third grader who was re- 
quired to attend Summer Bridge was 2.0 
(see Figure l-4a). Thus, in order to be 
promoted, on average, a third grader had to increase 
her test score by 0.8 GEs, or 8 months, during the six- 
week program. Unfortunately, on average, these stu- 
dents increased their reading test scores by only three 



FIGURE 1-4A 

Average Second- and Third-Grade Reading Scores for 
Third Graders Required to Attend Summer Bridge 
Compared to All Third Graders 
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months during the prior school year. We see a similar 
pattern in other grades (see Figures l-4b and l-4c). 
The average sixth grader in Summer Bridge in 1997 
needed to raise her test score by nine months and 
an average eighth grader by over a year (11 months) 
to meet the cutoffs. Thus, Summer Bridge students in 
1997 entered the program needing substandal test-score 
increases in order to meet the promotional cutoffs. 

Some of the very low test scores and small gains 
during the school year among students who needed to 
attend Summer Bridge were due to the fact that stu- 
dents were evaluated using only one score that may 
have been the result of a bad testing day (we address 
this problem in Chapter 2). Students who had abnor- 
mally low test scores might exaggerate the observed 
overall achievement gaps of students who failed to reach 
the cutoffs. Looking at their pre-promotional gate 
grade test scores (fifth grade, for example), we see that 
this group was at substantial risk the year before as 
well. The average sixth grader who was required to 
attend Summer Bridge entered sixth grade with read- 
ing scores over a year and seven months (1.7 GEs) 
below grade level, over a year below the average CPS 



student. Thus, not only did Summer Bridge students 
have very poor test performance, on average, in read- 
ing at the end of sixth grade, but these students were 
also, as a group, consistently behind. 

The test-score gap faced by Summer Bridge students, 
decreased somewhat between 1997 and 1999. In 1999, 
the average May test score of third and sixth graders in 
Summer Bridge was two months higher than in 
1997 — meaning that students in these years were, on 
average, two months closer to the test -score cutoff on 
entry into Summer Bridge. This was also true in eighth 
grade, despite the yearly increase of the promotional 
test-score cutoffs. Thus, improvements in performance 
during the school year both reduced the number of 
students who were required to attend Summer Bridge 
and left those students who failed to meet the promo- 
tional cutoffs during the school year with a smaller 
test-score gap to close over the summer. 

Throughout this report, we focus more on read- 
ing than mathematics. We do so because most stu- 
dents attend Summer Bridge because they fail to 
meet the cutoff in reading. Over 85 percent of third 
and sixth graders and over three-quarters of eighth 
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FIGURE 1-4B 

Average Fifth- and Sixth-Grade Reading Scores for 
Sixth Graders Required to Attend Summer Bridge 
Compared to All Sixth Graders 



FIGURE 1-4C 

Average Seventh- and Eighth-Grade Reading Scores 
for Eighth Graders Required to Attend Summer 
Bridge Compared to All Eighth Graders 
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graders attend Summer Bridge because they failed to 
meet the promotional criteria in reading or in both 
reading and mathematics. The proportion of students 
meeting the promotional criteria in mathematics in- 
creased between 1997 and 2000 so that, particularly 
in the third and sixth grades, increasing proportions 
of Summer Bridge students had already met the test- 
score cutoff in mathematics (see Figures l-5a, l-5b, 
and l-5c). 

Once in Summer Bridge, students receive instruc- 
tion in both subjects regardless of which cutoff they 
missed. In a previous report, we found that students 
who had to raise their test scores in both reading and 
mathematics had less success in meeting the test-score 
cutoffs in Summer Bridge than those who had to raise 
their test scores in only one subject.^ The increasing 
numbers of students meeting the mathematics cutoffs 
resulted in a drop in both the proportion of students 
who had to attend Summer Bridge for mathematics 
and the proportion who had to attend for both math- 
ematics and reading. By 2000, over half of third- and 
sixth-grade Summer Bridge students needed to raise 
their test scores in reading only. In 2000, the training 
for Summer Bridge teachers included the recommen- 
dation that teachers take into account that, in many 
cases, the majority of their students would need to 
meet the cutoffs in reading but not necessarily in math- 
ematics. In eighth grade, however, we see a different 
trend. Raising the cutoffs increased the number of 
eighth graders who needed to raise both their reading 
and mathemacics scores for promotion. 

How Many Students Actually Attend 
Summer Bridge? Estimating Testing Rates 

So far, we have described the characteristics of CPS 
students who are required to attend Summer Bridge, 
or, in other words, those students who did not meet 
the promotional criteria in the spring. An important 
question in both evaluating the role of summer pro- 
grams under high-stakes testing and interpreting re- 
sults is: How many of these students actually attended 
the program? 



FIGURE 1-5A 

Almost All Summer Bridge Third Graders Needed to 
Improve Their Reading Scores 

Third Grade, 1997 and 2000 
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pass math pass reading only only subjects 
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The test*score cutoff for 3rd grade was 2.8 in 1997 and 2000. Percentages are 
^r included students only. 

rhe CPS Summer Bridge policy states that atten- 
dance is mandatory for students who do not meet the 
test-score cutoffs in the spring of the promotional gate 
grades. Students who miss three or more days of the 
program are not supposed to be allowed to take the 
ITBS in August, although this aspect of the policy^ may 
not be enforced evenly across schools. Although teach- 
ers in Summer Bridge take attendance, those records 
are not centrally located. Thus, we do not Icnow ex- 
actly how many students who fail to meet the cutoffs 
actually go to Summer Bridge, nor do we know what 
the absence rates are for students who attend. 

We do know, however, which students failed to meet 
the test-score cutoffs at the end of the school year and 
which students took the ITBS at the end of the sum- 
mer. If we assume that most teachers enforced the at- 
tendance policy, we can gain an estimate of 
participation in the program by comparing the num- 
ber of students who failed to meet the test-score cut- 
offs in spring with the number of students who took 
the ITBS after Summer Bridge. We call this the “test- 
ing rate.'’ 

Using this method we see that, in each year, over 
80 percent of students who did not meet the test-score 
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FIGURE 1-5B 



About Half of Summer Bridge Sixth Graders 
Needed to Pass Mathematics 

Sixth Grade, 1997 and 2000 
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■ 1997 01 2000 

In 1997, the test-score cutoff for 6th grade was 5.3. In 2000. it was increased to 5.5. 
Percentages are for included students only. 



FIGURE 1-5C 

As the Test-Score Cutoffs Were Raised, More Eighth 
Graders Needed to Pass Both Subjects 

Eighth Grade, 1997 and 2000 




pass math pass reading only only subjects 



a 1997 n 2000 

In 1997, the 8th grade test-score cutoff was 7.0. By 2000, it had increased to 7.7. 
Percentages are for included students only. 



cutoffs in spring took the ITBS in August (see Figure 
1-6). This method, however, most likely underesti- 
mates participation in the program. First, some stu- 
dents who fail in spring may leave the school system 
altogether by moving to another school district or 
transferring to a private school. Second, some students 
may receive 'waivers” in June, meaning that because 
of extenuating circumstances, their schools exempted 
them from the policy and from attending Summer 
Bridge. Third, some students may not have been re- 
quired to attend Summer Bridge if, at the end of the 
school year, they were in the process of being placed 
in special education. About half of students who failed 
to meet the ITBS cutoffs in spring and did not talce 
the test in August left the school system, moved into 
special education, or were promoted (despite not meet- 
ing the test-score cutoffs or attending Summer Bridge) 
in the next school year. We calculated an 'adjusted 
testing rate” in Summer Bridge accounting for these 
legitimate reasons for not attending the program. The 
adjusted rates suggest that in 1999, over 90 percent of 
students who were required to attend Summer Bridge 
and did not fall into one of the three exclusion catego- 
ries described above were tested at the end of the sum- 



mer. These testing rates have been relatively constant 
across years. I’hus, a very high proportion of students 
who are required to attend the Summer Bridge pro- 
gram are tested, although we do not have data on how 
often these students attended. 

In addition to looldng at overall participation rates, 
we examined how adjusted testing rates varied by 

FIGURE 1-6 

Testing Rates in Summer Bridge Were High 




0 20% 40% ■ 60% 80% 1 00% 

^ Percent of students who foiled the ITBS in spring and were 
then tested in Summer Bridge 

M Adjusted percent tested: removing those students who 
transferred out of CPS, were promoted at the end of the 
summer despite not attending Summer Bridge, or were 
placed in special education the next foil 

Numbers are for 1999. Patterns for other years are similar. 1,506 third graders did 
not meet the cutoffs in spring 1999 and did not have summer scores. The next year, 
190 of these students attended a school outside of the system, 91 were moved to a 
special education classroom, and 395 were promoted without having participated in 
Summer Bridge or meeting the ITBS cutoffs (these promotions are called "waivers"). 
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An average class size of 16 stu- 
dents means that most students 
and teachers experience dramatic 
reductions in class size betweem 
Summer Bridge and the regular 
school yean 



students* academic risk, gender, and race and eth- 
nicity. Our analysis found that August testing rates 
were almost identical across grades, by gender, and 
between racial and ethnic groups. In the third and sixth 
grades, students with the lowest test scores (high- 
risk students) did have slightly lower testing rates 
in Summer Bridge, although this difference was not 
substantial. 

Programmatic Characteristics of Summer Bridge 

The purpose of Summer Bridge is to provide students 
extra instructional time in reading and mathematics 
for those content areas that are tested on the FI BS. 
Third and sixth graders attend the program for diree 
hours per day for six weeks — a total of 90 hours of 
instruction. Eighth graders attend four hours per day 
for seven weeks, for a total of 140 hours of instruc- 
tion. Schools are given no flexibility in the design ol 
the curriculum; teachers are expected to teach a pre- 
scribed curriculum that is aligned with the ITBS and 
are provided all needed classroom materials in order 
to enact it. 

Student Assignment, Class Size, and 
Staffing Characteristics 

Initially, Summer Bridge was designed so that all stu- 
dents would attend the same school that they attended 
during the school year.^ During the program s first two 



years, CPS provided transportation for students whose 
home schools were not their neighborhood schools. 
In 1999, CPS changed diis policy because of budget- 
ary concerns, and students were required to attend their 
neighborhood elementary school for Summer Bridge 
regardless of whether they attended that school dur- 
ing the year. This policy change resulted in a small 
drop in the number of students attending their regu- 
lar school for the summer program. In 1999 and 2000, 
over 80 percent of Summer Bridge students attended 
the same school during the summer as in the regular 
school year. 

A central programmatic characteristic of Summer 
Bridge is its small class size. Schools are provided fund- 
ing for one teaching position for every 18 students. 
On the 1 999 Summer Bridge survey, we asked teach- 
ers to report their class size. Teachers reported, on av- 
erage, class sizes of 16 students (see Figure 1-7). Close 
to half of teachers (46 percent) reported that their class 
size was between 13 and 17 students. Twenty-four 
percent reported particularly small class sizes (below 
1 2 students), and 30 percent of teachers reported class 
sizes over 18. There was negligible difference across 
grades and schools in teachers* reports on class size. 

An average class size of 1 6 students means that most 
students and teachers experience dramatic reductions 
in class size between Summer Bridge and the regular 
school year. According to contractual agreement, el- 
ementary schools receive funding for one classroom 
teaching position per 29 students in kindergarten 
through third grade and one position per 31 students 



FIGURE 1-7 

The Average Summer Bridge Class Size was 16 in 1999^^ 
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* As reported by teachers on the 1999 Summer Bridge survey. 
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An important question to consider 
in evaluating the effectiveness of 
Summer Bridge is: How much does 
the program actually cost? Ex- 
penses for Summer Bridge can be 
broken down into three categories: 
teachers and materials, administra- 
tion, and operations (transportation 
and security). 

1. Teachers and materials. 

Schools are provided funding for 
one teacher for every 18 students, 
although the program will consider 
funding a position if there are fewer 
students.’ Teacher pay is funded at 
an average estimated cost of $42 
per hour for four hours per day over 
six weeks for third- and sixth-grade 
teachers (individual teacher pay var- 
ies by years of experience, years of 
education, etc.). Eighth-grade 
teachers are paid for 4.5 hours per 
day over seven weeks. Teachers are 
also paid for 12 hours of profes- 
sional development. Thus, the av- 
erage third- or sixth-grade Summer 
Bridge teacher is paid $5,544 for 
teaching in the program, and the av- 
erage eighth-grade teacher earns 
approximately $7,119. If there are 
an average of 1 6 students per class, 
then the cost of a Summer Bridge 
teacher is approximately $347 per 
student in the third and sixth grades 
and $445 in the eighth grade. The 



The Cost of Summer Bridge 

Chicago Public Schools (CPS) Office 
of Schools and Regions provided an 
estimate of the costs of curricula, ma- 
terials, and supplies per student.^ On 
average, between 1 997 and 2000, the 
cost of ail materials for Summer 
Bridge was $1 36 per student for third 
and sixth graders and $171 for eighth 
graders. Using these data, we can es- 
timate that the average cost per stu- 
dent for Summer Bridge teachers and 
materials was around $528. At the 
high end, adding costs for adminis- 
tration and operations would increase 
the cost of the program by approxi- 
mately $100 to $628 per student. 

2. Administration. Schools are 
funded for administrative support for 
Summer Bridge based on the size of 
all summer programming in that 
school. If a school has six to 1 2 class- 
rooms in summer, one teacher is paid 
for two hours of administrative work 
per week. If there are over 12 class- 
rooms, that school receives funding 
to pay one teacher as a summer ad- 
ministrator. in addition, schools re- 
ceive $2,500 per summer to fund 
secretarial needs regardless of 
whether or not they have a Summer 
Bridge program. There are approxi- 
mately 48 regular CPS teachers hired 
each year as monitors or coaches for 
the program. They work six hours per 



day for eight weeks at a total cost 
of approximately $500,000. In ad- 
dition, teacher training for Summer 
Bridge costs about $1 million per 
year. Also, high school and college 
students who are hired as Summer 
Bridge tutors are paid approxi- 
mately $10 per hour. 

3. Operations. Additional costs for 
Summer Bridge include funds for 
security and transportation. Secu- 
rity costs total approximately 
$200,000 per summer. In its first 
two years, transportation for Sum- 
mer Bridge cost approximately $1 
million per summer. However, in 
1 998, CPS instituted a policy requir- 
ing most students to attend their 
neighborhood schools for Summer 
Bridge, reducing transportation 
costs to about $200,000. 

All of the costs listed above are di- 
rectly related to running Summer 
Bridge and do not include indirect 
costs that come with keeping the 
school buildings open during sum- 
mer (e.g., utility costs such as air 
conditioning), the cost of adminis- 
tering the Iowa Tests of Basic Skills 
during the program, and the cost 
of initially planning and staffing the 
program, which includes identify- 
ing students who need to attend and 
informing students and parents. 



' CPS decides whether or not a school will have Summer Bridge based on the number of students who need to attend within each grade in the school. 
For example, if one school's third grade has approximately zero to six students who need to attend Summer Bridge, that grade will not have a 
program. If there are seven to 12 students who need to attend, then CPS will consider funding Summer Bridge for that school's third grade (taking 
into account whether the school has Summer Bridge in other grades, how easily students could be clustered in another school, etc.). If there are 13 
or more students who need to attend, the school will have a program for that grade. Students who need to attend Summer Bridge but whose schools 
do not offer a program are sent to nearby schools, 

^Estimates of Summer Bridge costs were obtained from the CPS Office of Schools and Regions through an interview with William McGowan, Director 
of Schools and Regions, conducted In spring 2002. 
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in the fourth through eighth grades.^ In addition, 
about half of Summer Bridge teachers responding 
to the 1999 survey reported having tutors available 
to assist with instruction. 

An important characteristic of Summer Bridge is 
that, whenever possible, teachers in the program are 
regular CPS classroom teachers who are paid at union 
rates over the summer. Principals are responsible for 
recruiting teachers for the program. We know little, 
however, about what might be the most effective 
method of staffing programs like Summer Bridge. On 
the one hand, students may benefit from the continu- 
ity of having the same teacher during the school year 
and the summer — a teacher who knows them and 
is familiar with their behavior and learning needs. 
On the other hand, students may benefit from a 
fresh start with a new teacher, particularly if they 
did not experience great success with their teacher 
during the school year. 

Our 1999 survey of Summer Bridge teachers al- 
lowed us to examine how principals staffed Summer 
Bridge classrooms. In 1999, 87 percent of third- and 
sixth-grade teachers reported teaching Summer Bridge 
in the same school in which they taught during the 
school year (see Figure 1-8). In addition, over 80 per- 
cent of Summer Bridge teachers were regular classroom 
reading and/or mathematics teachers during the year. 

Although most Summer Bridge teachers were regu- 
lar classroom teachers working in their regular schools, 
they were less likely to be teaching Summer Bridge in 
the grade they taught the previous year. Of the three 
Summer Bridge grades, eighth-grade students were the 
most likely to be taught by eighth-grade teachers. Over 
half (54 percent) of eighth-grade Summer Bridge teach- 
ers reported that they taught eighth grade during the 
previous school year (see Figure 1-9). However, only 
about one-third of third- and sixth-grade Summer 
Bridge teachers (36 percent and 31 percent, respec- 
tively) reported teaching that grade the previous year. 
Most third-grade teachers were drawn from the pri- 
mary grades (57 percent came from second through 
fourth). Similarly, sixth-grade teachers were drawn pri- 
marily from the fifth through eighth grades (67 per- 



FIGURE 1-8 

The Majority of Summer Bridge Teachers Taught 
Reading and/or Mathematics during the 
School Year and Taught in the Same School 
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These figures are based on teacher reports on the 1999 Summer 
Bridge survey. 



FIGURE 1-9 

One-Third to One-Half of Summer Bridge Teachers 
Taught in the Same Grade during the School Year 

Based on the 1999 Summer Bridge Teacher Survey 
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cent). It is possible that the smaller proportion of third- 
grade teachers we found teaching third-grade Sum- 
mer Bridge may be due . to there being many more 
third-grade students in the program and principals may 
have had difficulty recruiting enough third-grade 
teachers for the available positions in that grade. Also, 
principals, out of concern about content knowledge 
or classroom management, may place more emphasis 
on recruiting same-grade teachers in the eighth grade 
than in the third or sixth grades. 

The fiict that fewer third- and sixth-grade teachers 
taught Summer Bridge in the same grade that they 
taught the previous year meant that teachers in these 
grades were much less likely to know their students. 
The 1999 Summer Bridge teacher survey asked teach- 
ers to report the number of students that they knew 
previously. The average third-grade teacher reported 
knowing only 36 percent of her Summer Bridge stu- 
dents, compared with sixth-grade teachers knowing 
43 percent, and eighth-grade teachers knowing over 
half (54 percent). Over twice as many eighth-grade 
teachers dian third-grade teachers (4 1 percent com- 
pared to 18 percent) reported knowing almost all (80 
to 100 percent) of their students before Summer 



FIGURE 1-10 

Eighth-Grade Summer Bridge Teachers Knew More of 
Their Students Than Third- and Sixth-Grade Teachers 
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Over twice as many eighth- 
grade teachers than third-gmde 
teachers (4l percent compared 
to 18 percent) reported know- 
ing almost all (80 to 100 per- 
cent) of their students before 
Summer Bridge. 

Bridge (see Figure 1-10). In Chapter 5, we examine 
how these differences in staffing characteristics — 
whether teachers taught in the same school and in 
the same subject, and the percentage of the students 
they knew before the summer — are associated with 
student outcomes. 

The Summer Bridge Curriculum 
Instructional Foci 

As its name suggests, the Iowa Tests of Basic Skills, or 
n BS, is a basic skills test. The reading portion of the 
ITBS focuses on- testing reading comprehension skills 
in three areas: factual meaning, inferential and inter- 
pretative meaning (e.g., drawing conclusions or infer-, 
ring the traits or feelings of a character), and evaluative 
meaning (e.g., identifying the main idea of a para- 
graph) . The mathematics portion of the ITBS covers 
problem-solving slfills, data interpretation, and math- 
ematical computation. The Summer Bridge curricu- 
lum was designed so that the instructional emphasis is 
approximately proportional to the emphasis of those 
topics on the ITBS. This does not vary substantially 
from grade to grade, as is illustrated in a comparison 
across third- and eighth-grade reading. At each grade 
level, the percentage of time devoted to topic areas in 
the Summer Bridge curriculum roughly corresponds 
to the percentage of questions in that topic area on 
the ITBS. Figure 1-1 1 demonstrates this by com- 
paring the percentage of questions on the ITBS (as 
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FIGURE 1-11 

Instmctional Foci from the Summer Bridge Reading Curricuium 
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Analysis was conducted on the 1999 Summer Bridge curriculum. For reading, we report the breakdown of questions on the ITBS as provided in the 
Summer Bridge Curricuium. For math, we referred to the booklet titled "Iowa Tests of Basic Skills: Content Classifications with Item Norms." Percent 
time listed as instructional focus was obtained from a count of the proportion of days in which the focus is listed. When there were multiple fod, a 
weighted average was calculated. See Appendix A for sixth-grade reading and third>. sixth-, and eighth-grade mathematics instructional foci. 
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reported in the Summer Bridge curriculum) in each 
topic area and the percentage of time that topic area 
was listed as a core instructional focus oi a days lesson.^ 

The reading curriculum in Summer Bridge relies 
upon a series of readings, text and skill books, and 
classroom libraries. The curriculum does not, in gen- 
eral, introduce skills as discrete topics. Rather, it de- 
velops a set ot reading comprehension concepts (main 
idea, inference) and sldlls by repeatedly using a variety 
of reading materials over the course of the summer. 
For example, students are expected to apply the skills 
needed to infer the traits, feelings, or motivation of 
characters to virtually every story they read. Figures 
l-12a through l-12c. provide examples of typical 
lesson plans from the Summer Bridge reading curricu- 
lum (see Appendix B for example.s of daily mathemat- 
ics lesson plans). One lesson was selected from each of 
the three grades, and days with similar instructional 
foci were chosen.'' 

The Summer Bridge curriculum is described as be- 
ing at “grade level” because it focuses on die skills tested 
on the ITBS for each respective grade. However, as 
seen in Figure 1-11, and in Appendix A, the instruc- 
tional foci do not change dramatically across grade lev- 
els. Nonetheless, there are two ways that the Summer 
Bridge curriculum does differ across the three grades: 
differences in the complexity and difficulty of the texts 
and problems to which students are applying concepts, 
and differences in the level at which students are ap- 
plying each reading and mathematics concept. For 
example, in reading, students are asked to apply simi- 
lar concepts to “grade level” reading material in all three 
grades, but both the variety and complexity of the 
materials used increase across grades. The third-grade 
reading curriculum uses both the smallest number and 
variety of materials. In the third grade, almost all of 
the' students’ reading materials are brief stories that 
are linked to a central theme, such as courage. Third 
graders are also exposed to several poems and one ar- 
ticle. In sixth grade, readings include a number of short 
stories, passages that are more informational in na- 
ture, and several articles. In the eighth-grade curricu- 



lum, students read the broadest range of materials, in- 
cluding several stories, a novel of students’ choice, a 
number of poems, and a number of articles. 

Instructional foci are also intended to be applied 
“at grade level,” and thus in a more challenging way in 
the sixth and eighth grades than in the third. Students 
in higher grades are asked to interpret texts in more 
sophisticated ways. For example, in applying the con- 
cept of identifying the author’s viewpoint, third-grade 
students receive instruction on specific subjects, in- 
cluding identifying the narrator of a story, defining a 
point of view, considering their own points of view, 
and describing a narrator’s identity and attitude. The 
eighth-grade curriculum specifies instruction that in- 
cludes practice in identifying different points of view, 
acknowledging and understanding that authors ma- 
nipulate their readers, and instruction on the fact 
that the tone of a narrative can influence readers. 

Each lesson of the Summer Bridge reading curricu- 
lum is divided into a teacher-directed activity, tuto- 
rial/skills development, and homework. The 
teacher-directed activity is the primary time when in- 
struction takes place. The curriculum designates 50 
minutes to teacher- directed activities and 50 minutes 
to tuto rial/ skills development. Ten minutes are devoted 
to homework instruction. Teachers are given wide flex- 
ibility in how to organize the teacher-directed activity 
and, as we will see in Chapter 5, teachers enact the 
Summer Bridge curriculum in a wide variety of ways, 
providing instruaion that ranges from highly individu- 
alized and targeted, to meeting individual students’ 
needs, to simply following the curriculum verbally. 

The tutorial/skills development activities provide 
structured directions to teachers regarding which ma- 
terials to use. Despite several exceptions in which stu- 
dents were asked to write a paragraph or construct a 
piece of their own work, most of the tutorial/skills 
development activities were devoted to reading short 
passages and doing multiple choice exercises such as 
working out of workbooks or using SRA materials. 

To summarize, the primary focus of the Summer 
Bridge reading curriculum is on developing reading 
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FIGURE 1-12A 



Lesson: 16 

Sample Daily Lesson Plan from the 
Summer Bridge Curriculum, 1999 

Third Grade Reading 



Instructional Focus 



Draw conclusions, make inferences, deduce meanings 
not explicitly stated In text. 



Teacher-Directed Activity 



Check and review homework. Place the instructional focus on 
the chalkboard and briefly discuss with the students. Invite the 
students to help set a purpose for reading the story. 

Ask: What do you think we could find out if we read the story? 
Record their responses to this question and suggest that they 
locate the specified information. Turn to page T416. Help the 
students formulate strategies to find out the problem Maria 
Isabel is having at school. Ask the students to list the reasons 
the author gives for Maria’s problem. Have the class form 
cooperative groups of three or four and brainstorm other ways 
Maria could have informed her parents of her problem at 
school. Invite the class to read these pages orally in their small 
groups. Encourage the students to tell what they notice about 
each other's reading. Ask students the informal assessment 
question under “Drawing Conclusions” on p. T419. Use the 
Teach/Model graphics to help the students think about story 
evidence and what they already know. Explore the meaning of 
story evidence with the students so that they will be able to find 
support for their answers. 



Tutorial/Skilt Development 



Distribute SRAs. Tell the students that today’s listening activity Is 
a story from Greece. Continue with the introduction and listening 
story, following the teacher's edition, pp. 37-43. 



Homework 



Assign “Bird Lady" from Focus on Reading. Have the students 
view the photographs and speculate on what the title infers. 

Read p. 70 to draw the class into the story. Direct the students to 
complete the story and the activities on pp. 72-73. 



Source: Chicago Public Schools 1999 Summer Bridge Teacher 
Handbook, Grade 3. The lesson has been condensed. 



FIGURE 1-12B 



Sample Daily Lesson Plan from the 
Summer Bridge Curriculum, 1999 

Sixth Grade Reading 



Instructional Focus 



Draw conclusions. Apply information from text to new situation; 
depict information in new form. Deduce meanings not explicitly 
stated in text. 



Teacher-Directed Activity 



Review and discuss homework assignment from the previous day. 
Place the instructional focus on the chalkboard and discuss briefly 
with the students. Explain to the students that when they create 
something— a report or a painting — or make a decision for 
drawing a conclusion about something, they combine information 
that they have learned from outside sources with their own 
knowledge and experience. Have the students reread the first 
paragraph on p. 478. Discuss how Margaret had to gather and 
combine information from different sources. Model the lesson 
T1004 with the students. Draw on the chalkboard the diagram 
found on p. R59 and discuss with the class. Have the students 
look at the photograph Basket of Light on p. 485. use the question 
found in the visual/spatial model on p. R59 (to reinforce the 
process for gathering and combining information to reach a 
conclusion). 



Tutorial/Skill Development 



Assign Focus on Reading, pp. 58-61 , to students who need 
further practice in the focus skill for the day, drawing conclusions. 
Work through the exercises with students while monitoring their 
progress. 

Use the SRA Reading Laboratory 2c to provide further 
development in reading comprehension. Power Builders at the 
appropriate color should be inserted in the students' record books 
in advance. Check students’ booklets for progress. 



Homework 



Assign Signatures Practice Book, p. 144. Explain the activity to 
ensure understanding. Go over the exercise together. 



Source: Chicago Public Schools 1999 Summer Bridge Teacher Handbook. 
Grade 6. The lesson has been condensed. : 
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FIGURE 1-12C 



Sample Daily Lesson Plan from the 
Summer Bridge Curriculum, 1999 

Eighth Grade Reading 



Instructional Focus 



Draw conclusions. Make inferences. Deduce the meanings that 
are not explicitly stated in the text. 



Teacher«Dlrected Activity 



Review homework. Place the instructional focus on the 
chalkboard and discuss briefly. Invite students to share their plot 
summaries with the class, explain their poster designs, or read 
their letters written to one of the characters from the story. Display 
the best student work in the classroom. Encourage the class to 
discuss whether the homewori< directions were followed. 

Have volunteers share a recent incident that they can recall which 
Involved drawing a conclusion. Encourage ideas by saying, "if you 
see two people greet each other with smiles and a handshake, 
you will readily conclude that they are friends or they are happy to 
see each other." Listen to the examples and discuss. Introduce 
the assigned selection by reading orally and discussing the 
background information on p. 73 of Be a Better Reader in order to 
maximize comprehension. Preview the selection by studying the 
graphs and their interpretation. Read the tan boxes and discuss 
meaning that is not in the text. 

Allow students time to read silently the selection, Trends in the 
Work Force: 1940's-1950's,” pp. 73-75. Elicit volunteers to read 
passages orally in order to check for comprehension and word 
punctuation. Do "Interpreting Facts," pp. 75 and 76, as group 
exercises, discussing responses for each question before moving 
on to the next. Note areas of difficulty for further skill 
development. Have students answer selected questions from 
"Skill Focus" and "Real Life Connections," pp. 76 and 77. 



Tutorial/Skill Development 



Prepare the SRA Reading lab 3a assignment for each student. 
Insert the Power Builder at the appropriate level {brown or tan) in 
the Student Record Book. After the Student Record Books 
containing Power Builders have been passed out, ask students to 
turn to p. 21; follow the directions on p. 33 of the Teacher’s 
Handbook. 



Homework 



Instruct students to turn to p. 147 in Be a Better Reader. Explain to 
students that pp. 147-149 will reinforce their skills in using a 
dictionary. Students should have time to preview the lesson so 
questions can be answered. The pages should be completed for 
homework. 



Source; Chicago Public Schools 1999 Summer Bridge Teacher Handbook, 
Grade 8. This lesson has been condensed. 



comprehension skills rather than teaching more basic 
reading skills such as decoding. The Summer Bridge 
mathematics curriculum is focused on problem solv- 
ing, data interpretation, and computation, providing 
instruction both in the application of basic mathemat- 
ics computation skills and in analysis and estimation. 

What does it mean to say that the Summer Bridge 
curriculum is “test prep”? An important question is 
whether test- score increases that occur after the intro- 
duction of high-stakes testing reflect real achievement 
gains. These concerns are relevant when evaluating 
programs like Summer Bridge because the primary 
purpose of the program is to prepare students to raise 
their scores on a particular test. Test preparation can 
occur in several ways. The first way, which we term 
‘content alignment,” occurs when students are taught 
specific material that will be on the test. A second way, 
which is also the most direct, occurs when teachers 
spend time teaching students strategies for answering 
questions and giving practice tests. And, a third ap- 
proach, referred to as “assessment alignment,” occurs 
when students are asked questions or assessed in ways 
that give them practice in the particular form in which 
the test assesses knowledge. For example, the ITBS is 
a multiple-choice test. Students might, in general, be 
able to express the main idea of a paragraph but would 
be more likely to get that answer right if they practice 
translating that knowledge into a correct answer in a 
multiple-choice format. Each of these techniques — 
content alignment, test strategies, and assessment align- 
ment — would likely lead students to get higher scores 
on one particular test because they are taught what is 
on thattest 'm ways that the test measures performance. 

The close alignment between the ITBS and the 
Summer Bridge curriculum means that the Summer 
Bridge curriculum can be characterized as test prepa- 
ratory because students are being taught the specific 
skills on which they will be tested. The extent to 
which instruction in Summer Bridge also empha- 
sizes direct test-taking strategies is less clear. The 
material for Summer Bridge includes the Test Best 
book, the test preparation manual that accompa- 
nies the ITBS. And, the teacher s handbook for the 
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Is Summer Bridge Just Test Prep? 

One of the first questions that inevitably arises about intensive academic programs linked to high-stakes testing 
is: Is it all just test prep? We note that test preparation can be defined in several ways, including teaching 
students strategies for answering questions on standardized tests, teaching students material that will be cov- 
ered on the test, and having students practice skills in the format in which they will be tested (e.g.. practice 
tests, and multiple-choice exercises). We found very little evidence of the first type of test preparation in Sum- 
mer Bridge. In all grades and in both reading and mathematics, teachers spent less than 1 percent of their time 
explicitly teaching students test preparation strategies, coaching them on taking the Iowa Tests of Basic Skills 
(ITBS), and giving them advice on how to take the test. 

On the other hand, almost all of the content taught In Summer Bridge is material that is covered on the ITBS. 
Given that the goal of Summer Bridge is to provide students with an opportunity to gain skills that will increase 
their test scores, the fact that the content is highly aligned with the ITBS and that the vast majority of teachers 
taught that content during lessons is to be expected. Finally, many of the materials used in Summer Bridge (e.g., 
SRA) are constructed in a multiple-choice format. Often, when students were engaged in skills practice, they 
were reading paragraphs and answering multiple-choice questions or solving basic math problems. Also, we 
found that in our sample of classroom observations, teachers spent about 4 percent of class time having stu- 
dents take tests or quizzes (see Chapter 5 for further details and Appendix I about methodology.) This includes 
taking practice tests using books like Test Best, a practice test book geared towards the ITBS. So, although we 
saw very little explicit coaching or teacher practice geared toward helping students learn strategies for taking 
the test (for example, process of elimination for multiple-choice questions), the entire program was definitely 
geared towards teaching students both the content and the skills tested on the ITBS. 



reading curriculum specifically notes that “during 
their regular lessons, whenever possible, teachers should 
incorporate the test question prompts which are in- 
cluded in the test- preparation booklet.''^^ At the same 
time, day-to-day lesson plans in reading place little 
emphasis on traditional test preparation activities such 
as taking practice tests, working in test practice books 
designed to teach test- taking strategies, or drilling on 
the specific method of answering questions on a par- 
ticular test. The Test Best book is primarily for skills 
practice and is used on only three to four days in the 
third-, sixth-, and eighth-grade reading curricula. 

The mathematics curriculum places much more 
emphasis on direct test preparation. In the sixth and 
eighth grades, the mathematics curriculum provides 
one day a week of explicit test preparation, usually on 
Fridays, where, according to the Teachers Flandbook, 
“students in grades six and eight will be taught test- 
taking skills, using an approach that includes teacher- 



directed instruction, guided practice, and independent 
practice.” In third grade, test preparation occurs as an 
explicit instructional focus on approximately eight 
days, and a test preparation activity is often included 
as an additional skills activity for the end of the day. 
Finally, the materials and texts used for Summer Bridge 
do reflect an emphasis on assessment, particularly mul- 
tiple-choice questions (such as use of SRA cards and 
multiple-choice exercises at the end of the chapters in 
the reading curriculum). 

Summary 

The goal of this chapter was to describe the essential 
elements of Summer Bridge, both to understand the 
findings in the chapters that follow and to provide data 
for generalizing those results. This chapter focused on 
identifying critical characteristics of Summer Bridge 
students and the program itself that are relevant for 
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interpreting findings in subsequent chapters. Here we 
summarize what we think are the most important find- 
ings in this chapter. 

First, the Summer Bridge program is a large sum- 
mer program serving urban and predominantly mi- 
nority students. In the third grade, Summer Bridge 
students are predominantly African-American, and in 
sixth and eighth grades, they are predominandy Latino 
and African-American. Students who attended the pro- 
gram performed substantially below grade level on 
both the promotional grade’s test and the previous 
years test. Over the course of this study, students 
who attended Summer Bridge did so primarily be- 
cause of low reading scores. 

Second, a critical question is the extent to which 
students are self-selecting into summer programs and 
whether the programs are serving only the most moti- 
vated students. We addressed this issue by examining 
how many students who were required to attend Sum- 
mer Bridge were tested at the end of the summer. We 
found that, at over 80 percent, testing rates in the pro- 
gram were relatively high. When we excluded students 
who might not have attended Summer Bridge because 
they left the school system, were promoted, or placed 
in special education the next year, testing rates were 
over 90 percent. These testing rates are impressive for 
such a large summer program. They highlight both 
the importance placed on the program by the schools 
and the system, and the extent to which the threat of 
non-promotion may have increased participation. Test- 
ing rates at the end of the summer do not tell us about 
student attendance from day to day, but they do 
suggest that Summer Bridge results can be general- 
ized to more than just a small proportion of moti- 
vated students. 

An important component of Summer Bridge is that 
most students attend the same school for Summer 
Bridge as they did during the school year and are taugh t 
by a CPS teacher from their home school. For the most 
part, these aspects of Summer Bridge were maintained 
over the programs first four years. Even after the sys-- 
tem decided not to provide transportation for students, 



over 80 percent of Summer Bridge students attended 
the same school as during the school year. At the same 
time, most Summer Bridge teachers, with the excep- 
tion of eighth grade, were not teaching in the grade 
they taught during the school year. This meant that 
most Summer Bridge teachers did not know a high 
proportion of their students prior to the summer. 

Third, a central component of Summer Bridge is 
small class size. Schools are provided one classroom 
teacher for every 18 students, and teacher reports 
suggest that actual class sizes are slightly lower, on av- 
erage 16 students. These reported class sizes would 
make the average Summer Bridge class size substan- 
tially smaller than what students and teachers experi- 
ence during the school year. 

Fourth, a critical component of Summer Bridge is 
the mandatory, centrally developed curriculum. The 
reading curriculum in Summer Bridge focuses on 
building students’ reading comprehension skills as they 
are measured on the ITBS. The mathematics curricu- 
lum focuses on data analysis, estimation, and basic 
mathematics concepts. Finally, the Summer Bridge 
curriculum is designed to raise student achievement 
scores on the ITBS by providing them more practice, 
instruction, and exposure to the specific sets of skills 
that will be tested on the ITBS. In this respect, the 
Summer Bridge curriculum. can be characterized as test 
preparatory. The reading curriculum gives much more 
emphasis to building more general reading compre- 
hension skills than to teaching test- taking strategies 
or providing practice tests. The mathematics cur- 
riculum, however, is much more test focused and 
gives attention to direct test preparation and prac- 
tice activities. 

In conclusion, Summer Bridge is a large and ambi- 
tious program directing extensive resources to the 
systems lowest performing students. In the following 
chapters, we examine the impact of this effort by an- 
swering these key questions: Are there learning gains 
in Summer Bridge? What are students' and teachers’ 
perspecti ves about the program? And, how do students 
describe their experiences in the program? 
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T he goal of Summer Bridge is to provide students who do not meet the 
promotional cutoffs at the end of the school year with extra support 
and a second opportunity to be promoted. This chapter takes an in- 
depth look at the short-term effectiveness of Summer Bridge.^ Do stu- 
dents raise their test scores over the summer? And, are test-score gains large 
enough for students to be promoted to the next grade? We begin by exam- 
ining the average effects of Summer Bridge in the first four years of the 
program (1997-2000). We then examine two critical issues that arise in 
the use of summer interventions under high-stakes testing. First, is Sum- 
mer Bridge more effective for some students than for others, and, in par- 
ticular, how do program outcomes differ for students with very low test 
scores compared to those who are closest to the promotional cutoffs? And, 
second, to what extent might summer programs simply exacerbate exist- 
ing differences in school quality — do students in higher-performing schools 
benefit more than students in schools that are lower performing? 

Estimating Test-Score Gains in Summer Bridge: 

Adjusted Increases for 1 997-2000 

The simplest method of estimating test-score increases over the summer is 
to subtract a student s score at the end of the school year from her summer 
test score. This is the method that the school system uses when it reports 
test-score gains in Summer Bridge, The problem, however, is that because 
students are selected for the program on the basis of one test score, 
some will be required to attend because they had a bad testing day. To 
reduce the influence of random fluctuations in students’ test scores, we 
use a student’s entire test-score history to predict spring scores and 
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then subtract this predicted spring score from her ob- 
served summer score. We call this estimate of the ef- 
fect of Summer Bridge the adjusted Summer Bridge 
gain. This analysis is restricted to students who were 
first-time Summer Bridge participants, who were 
not retained the prior year, and who had to pass 
that portion of the test (reading or mathematics) to 
be promoted.^ 

How important is adjusting students test-score in- 
creases in Summer Bridge for regression to the mean? 
A comparison of observed and adjusted gains confirms 
that a simple comparison of spring and summer test 
scores would lead to substantial overestimates of im- 
provement on the ITBS in Summer Bridge. In 1997, 
for example, eighth graders who attended Summer 
Bridge increased their reading test scores by about a 
year (ten months) in the seven-week program (see Fig- 



ure 2-1). This is the estimate we would obtain by sim- 
ply subtracting observed summer test scores from ob- 
served spring test scores. Our adjusted estimate, which 
accounts for the fact that many of these students had 
below-average test performance in spring, is 6,5 
months (0.65 GEs). Although this is substantially 
lower than the observed gain, it is still very large. In 
fact, even after adjusting summer test-score gains for 
abnormally low spring test scores, estimates of test- 
score gains in the program are substantial and rela- 
tively consistent across the first three years. This was 
particularly true in the sixth and eighth grades. In the 
sixth grade, adjusted ITBS score increases in reading 
ranged from a low of 3. 1 months in 1998 to a high 
of 4.3 months in 1999 (see Figure 2-2a). Test-score 
gains were largest in the eighth grade. Between 1 997 
and 1999, the average eighth-grade Summer Bridge 



Estimating Student Test-Score increases in Summer Bridge 



A central focus of this chapter is to 
address the question of how much 
students' test scores increase dur- 
ing Summer Bridge. If the program 
were effective, one would expect 
students’ test scores to improve 
between spring and the end of sum- 
mer. This suggests that one might 
simply examine the change in test 
scores from spring to summer as 
an indicator of summer school 
learning. The problem with a simple 
comparison of scores is that stu- 
dents are selected for Summer 
Bridge by scoring below a specific 
cutoff. On any standardized test, 
there is considerable variation in 
student performance that is not re- 
lated to students' "true" achieve- 
ment level. For example, one 
student might be sick on a testing 
day or may accidentally miscode 
the answer key. Another may have 
a particularly good testing day or 



guess correctly on a question or two. 
This means that given two students 
with the same underlying achieve- 
ment levels, the one who has a bad 
testing day will be more likely to at- 
tend summer school. When this stu- 
dent is retested at the end of the 
summer, her score will likely increase 
simply due to chance even if the pro- 
gram had no impact. This regression 
to the mean artificially inflates sum- 
mer school learning gains. 

In this chapter, we correct for the 
problem of regression to the mean by 
using a student's learning gains over 
the course of his entire test-score his- 
tory to obtain a predicted spring test 
score. We obtain this predicted pre- 
Summer Bridge test score by estimat- 
ing a student growth model using a 
hierarchical linear model, or HLM (see 
Appendix C for a detailed description). 
This model uses a student's entire 



testing history and the learning 
trends of students in both pre- 
(1992-1996) and post-policy (1997- 
2000) cohorts. It is estimated sepa- 
rately for each grade (third, sixth, 
and eighth) and for each subject 
(reading and mathematics). The 
outcome variable is the student's 
test score in each year in that sub- 
ject on the Iowa Tests of Basic Skills 
(ITBS) measured in the grade 
equivalent (GE) metric. 

This procedure can be illustrated for 
a sixth-grade student; In May of 
sixth grade, this student had a very 
low test score on the ITBS. Her 
score was low both because she 
was a low-achieving student across 
her school career and because she 
had a bad testing day. If we use her 
prior test-score history to predict 
what her sixth-grade test score 
should have been given her prior 
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FIGURE 2-1 

Adjusted Test Score Gains Are Large, 
but Smaller Than Observed Gains 

1997 Observed and Adjusted Test-Score Gains in Summer Bridge 
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Student increased her ITBS reading scores, using the 
adjusted measure, by over six months. In the year 2000, 
adjusted Summer Bridge test-score gains declined in 
reading for eighth graders and for every grade in math- 
ematics (see Figure 2-2b). In the previous chapter, we 
also found that spring passing rates declined in 2000, 
particularly in the eighth grade. ^ Thus, not only did 
the students in these grades have poorer school year 
outcomes, but their Summer Bridge outcomes were 
also worse. 

In general, however, adjusted gains suggest that sixth 
and eighth graders who attended Summer Bridge ex- 
perienced substantial test-score increases over the sum- 
mer. Across the first four years of the program, the 
average sixth grader who attended Summer Bridge and 
needed to pass that subject increased her test scores by 
almost four months (0.39 GEs) in reading and three 



trend, we see that she still increased 
her test score in the summer, though 
not nearly as much as we would es- 
timate using her observed gain. We 
call this an adjusted gain. Using this 
methodology requires that we re- 
strict analysis to students for whom 
we have test scores prior to the 
promotional gate grade. This 
means that the analysis excludes 
students who are new to testing. 

A similar problem occurs when we 
try to compare summer gains to 
school year gains. The simplest 
method would be to compare learn- 
ing gains in summer to the gain that 
students made during the prior 
school year (for example, a student's 
spring to summer gain against the 
prior year's fall to spring gain). This 
is problematic for two reasons. First, 
if summer school students are those 



who, by definition, scored particularly 
low in spring of the prior year (given 
their "true" ability), the above method 
would artificially deflate the school 
year gains. Second, if the new pro- 
motional policy encourages students 
and teachers to work harder during 
the academic year, achievement gains 
during the promotional gate year will 



likely be larger than in the previous 
year. To address these problems, 
we also use students' estimated an- 
nual learning gains across their en- 
tire test-score history, obtained 
from the growth curve model, to 
measure the typical school year 
gains of students who attended 
Summer Bridge. 



Correcting for Regression to the Mean in Estimating 
Summer Bridge Test-Score Increases: An liiustration 
of a Sixth-Grade Student 
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FIGURES 2-2A AND B 

Adjusted Summer Bridge Test-Score Gains 

Reading, 1997 to 2000 



S-j Third grade 



Sixth grade 



Eighth grade 




^ ^ 





four months in reading for a program that lasts one 
and one-half months is impressive. It is hard to inter- 
pret these results without a comparison^ however. One 
approach is to compare Summer Bridge results to esti- 
mated test-score increases in other summer programs. 

Harris Cooper and his colleagues at the University 
of Missouri recently conducted an extensive review ol 
summer programs.'^ In order to compare test-score 
gains across summer programs that use different tests 
to measure student progress, they reported test-score 
gains in “effect sizes.” The effect size converts the 
achievement gain in the program into a scale based 
on the standard deviation of a one-year gain on that 
particular test.^ Effects sizes allow for direct com- 
parison of summer program gains across different 
achievement tests. 

Cooper and his colleagues estimated that students 
in remedial summer programs generally increase their 
test scores by 0.25 standard deviations (or by an effect 
size of 0.25)/’ Test-score gains in reading, however, tend 
to be smaller than in mathematics. They found an 
average effect size of 0.24 in reading, with a margin of 
error of 0.02 in either direction, and 0.27 in mathemat- 
ics, with a margin or error of 0.03 in either direction. 
They also found that remedial summer program effects 
tend to be larger for younger students than for students 
in the middle grades. 



and a half months (0.36 GEs) in mathematics (see 
Figure 2-3). Eighth graders, on average, increased their 
ITBS test scores by approximately six months in read- 
ing (0.59) and nearly five months (0.48) in mathemat- 
ics across the four years: Adjusted gains were smaller 
among third graders in both reading and mathemat- 
ics. Combining effects across years, adjusted gains for 
third graders were approximately two months in read- 
ing and three months (0.29) in mathematics. 

Summer Bridge Is Effective when Compared to a 
National Review of Other Summer Programs 

Given the length of Summer Bridge, these adjusted 
test-score increases appear quite large. An increase of 



FIGURE 2-3 

Eighth Graders Had the Largest Summer Bridge 
Test-Score Gains in Reading and Mathematics 

Average Adjusted Test-Score Gains 1997-2000 
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In order to compare the results of Summer Bridge 
to the findings of Cooper and his colleagues, we con- 
verted the estimated adjusted ITBS gains in Summer 
Bridge to effect sizes using the standard deviation of 
test scores on the ITBS.' Because adjusted Summer 
Bridge learning gains differed in 2000 from prior years, 
we show effect sizes for 2000 separately from those for 
the first three years of the program (1997-99, see Fig- 
ure 2-4a). The 1997-99 adjusted Summer Bridge read- 
ing test-score gain for eighth graders in 2000 is 
approximately four and a half months (0.44 GEs). 
Dividing this by the average of the standard deviation 
of ITBS scores for students in Summer Bridge, the 
0.44 GE increase translates into a 0.4 1 standard de- 
viation increase on the ITBS reading test. 

In the third grade, average effect sizes between 1997 
and 1999 in Summer Bridge are comparable to those 
found in other programs in reading and larger than 
other programs in math (see Figure 2-4b). In the sixth 
and eighth grades. Summer Bridge effect sizes were 
significantly larger than those found in other programs. 
In the eighth grade, Chicago’s Summer Bridge pro- 
gram produced average effect sizes that were twice that 
of other remedial summer programs. 

Test-Score Increases in Summer Bridge 
Versus the School Year: 

The Relative Efficienqf of Summer Bridge 

A second way to understand the magnitude of the test- 
score increases in Summer Bridge is to ask: Flow large 
were students’ ITBS test-score gains in Summer Bridge 
compared to their gains during the 37 weeks of the 
prior school year? 

There are six weeks of summer school for third 
and sixth graders and seven weeks for eighth grad- 
ers, There are also approximately six weeks between 
the administration of the spring ITBS and the end of 
the school year. If we assume that half of this period 
was dedicated to instruction, then third-grade students 



FIGURES 2-4AANDB 

Summer Bridge Had a Greater Effect in Sixth 
and Eighth Grade Than Other Summer Programs 

Reading Gains, 1997-1999 and 2000 
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Other Third grade Sixth grade Eighth grade 

programs* 



• Effect size for "other programs" from Cooper et al. (2000), page 71. 
The 95% confidence interval for this effect size Is 0.22 to 0.26. 



Mathematics Gains, 1997-1999 and 2000 
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The 95% confidence interval for this effect size is 0.24 to 0.30. 
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What Does Other Research Find on the Effects of Summer School? 



In Making the Most of Summer 
School: A Meta-Analytic and Narra- 
tive Review (2000), Harris Cooper, 
Kelly Charlton, Jeff Valentine, and 
Laura Muhlenbruck explored the ef- 
fects of summer school based on a 
review of 54 evaluations of summer 
school programs, 39 of which 
looked at academic outcomes. The 
research restricted analysis to pro- 
grams that were run during the sum- 
mer by a school district or university 
and had the goal of providing aca- 
demic support or remediation for 
students. The central findings of 
their analysis are the following: 

Remedial summer school pro- 
grams focused on lessening or re- 
moving learning deficiencies have 
a positive impact on student 
achievement. Cooper and his col- 



leagues estimated that students in 
these programs, on average, perform 
about one-fifth of a standard devia- 
tion higher (0.20 standard deviations) 
than control groups. These effects 
range from between one-seventh to 
one-quarter of a standard deviation 
on outcome measures. 

Summer programs have more posi- 
tive effects on the achievement of 
middle-class students than students 
from disadvantaged backgrounds. 

The authors found that studies look- 
ing at the effects of summer school 
on the achievement of students from 
middle-class families demonstrated 
effect sizes approximately twice as 
large as those for students from dis- 
advantaged families. One hypothesis 
for this effect is that summer pro- 
grams in middle-class school districts 



might have more resources, such 
as smaller class size or more struc- 
tured programs. Another hypoth- 
esis is that disadvantaged students 
and students in more disadvantaged 
school districts face greater ob- 
stacles to learning. 

Remedial summer programs have 
larger effects when run in a small 
number of schools or in smaller 
communities. The analysis finds 
that large summer programs may 
be more difficult to mount and run 
effectively, may have less quality 
control, and may give teachers and 
schools less ability to plan and tai- 
lor the program. At the same time, 
large programs tend to be run in 
more disadvantaged school districts 
and, thus, this finding may reflect 
the fact that large summer pro- 



who attended Summer Bridge would have had about 
nine weeks, or 2.25 months of instruction between 
spring and summer testing.’* The adjusted gain for 
third graders in 1997 was 2.5 months (0.25 GEs) in 
reading (see Figure 2-5). Thus, third graders who at- 
tended Summer Bridge in 1997 increased their test 
scores by approximately one month for each month 
of instruction, including the time between spring test- 
ing and the end of the regular school year. 

Did these students also improve their test scores at 
this rate during the school year? U.sing the same method 
that we used to calculate an adjusted gain, we esti- 
mated school year learning gains for Summer Bridge 
students based on their prior test-score trajectories. 
These results shed a different light on the relatively 
small learning gains for third graders in Summer 
Bridge. Although third graders who attended Sum- 
mer Bridge seemed to increase their test scores by one 



month for each month of instruction during the sum- 
mer, this was not the case during the school year. The 
average third grader who attended Summer Bridge in 
1997 gained roughly five months in reading, on aver- 
age, during the entire 37 weeks (10 months) of the 
prior school year. 

Sixth and eighth graders who attended Summer 
Bridge in 1 997 had larger average school year test-score 
increases than third graders. These increases, however, 
are still substantially below the average gains made by 
the typical CPS student. During the 1 997 school year, 
for example, the average eighth grader increased his or 
her ITBS scores by 1 .08 years in reading and fully 1.3 
years in mathematics, compared with school year gains 
for Summer Bridge students of 0.81 years, and 0.76 
years, respectively.’* Clearly, students who attend Sum- 
mer Bridge are those who struggled in previous years 
and had smaller learning gains than their classmates. 
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grams are often run in districts with 
less adequate resources that serve 
more economically and academi- 
cally disadvantaged students. 

Summer program effects are 
larger when there are small class 
sizes and when individualized in- 
struction is emphasized. Summer 
programs in which teachers used 
small groups or one-on-one instruc- 
tion produced the largest effects. 

Summer program effects tend to 
be larger in mathematics than in 
reading. This could mean that 
mathematics skills are easier to ad- 
dress in shorter time periods. It 
could also mean that because stud- 
ies have found that students lose 
more knowledge in mathematics 
than in reading over the summer, 



summer programs have a greater 
effect in mathematics. This is be- 
cause students generally may be 
better able to sustain or improve 
their reading than their mathemat- 
ics skills while out of school. 

Remedial summer school programs 
that serve primary and secondary 
students have larger impacts than 
programs serving middle-grade stu- 
dents. Differences in grade effects 
may be driven by across-grade dif- 
ferences in the instructional foci and 
content of summer school at differ- 
ent grades rather than by age-related 
differences. Cooper and his col- 
leagues' analysis found that teachers 
in early grade programs tended to re- 
port using the summer for more in- 
dividualized and creative instruction. 
Teachers in high school report equiva- 



lent academic content compared to 
the regular school year while teach- 
ers in middle school programs 
tended to emphasize more general 
study skills in content areas. 

Finally, Cooper and his colleagues 
did not find differences in the effects 
of summer school by whether stu- 
dents were required to attend the 
program or volunteered for the pro- 
gram. Attention to ensuring that stu- 
dents attend and that programs are 
of high quality does appear to be 
important. The researchers found 
that summer programs have aver- 
age to above average effects when 
they are structured so that instruc- 
tion is monitored and attendance 
is required. 



One way to compare Summer 
Bridge gains to school year gains is to 
convert ITBS gains into an efficiency 
ratio of gains per week in summer ver- 
sus during the .school year. In 1997, 
for example, the estimated test-score 
gain for third graders was 2.5 months 
for the nine weeks in the summer and 
5.2 months during the school year. 
Thus, the estimated gain per week for 
third graders in 1997 was 0.28 (2.5/9) 
in the summer and 0.1 4 (5.2/.37) dur- 
ing the .school year. Taking the ratio of 
these two, we find that third graders 
who attended Summer Bridge in- 
creased their test scores approximately 
two times faster per week in Summer 
Bridge than during the prior school 



FIGURE 2-5 

How Much Did Summer Bridge Students Gain during the School Year 
Versus the Summer? 

Adjusted School Year and Summer Gains 1997 




Third grade Sixth grade Eighth grade Third grade Sixth grade Eighth grade 



in Summer gains H SchooUyear gains 

Summer gains are adjusted. School-year gains are gains from the prior school year. 
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year (0.28/0.14 = 2, or 200 percent). An efficiency 
score of 100 percent would indicate that students in- 
creased their achievement test scores at roughly the 
same rate during the summer as the school year. A 
score of 200 percent indicates that students increased 
their test scores at a rate that was twice as fast in sum- 
mer than the school year. 

Across all three grades, efficiency ratios suggest that 
a week in Summer Bridge resulted in greater test- 
score increases than a week during the school year (with 
the exception of third-grade mathematics scores in sev- 
eral years where the ratio was approximately one to 



one). In reading, across all three grades, students' test 
scores increased almost twice as fast per week in sum- 
mer than the school year (see Figure 2-6a). Eighth- 
grade students in Summer Bridge increased their 
reading achievement at over three times the rate of the 
academic year. We see similar results in mathematics, 
except in the third grade, where gains were very high 
in 1997 but substantially lower diereafter (see Figure 
2-6b). It is important to recognize that high ratios in 
the sixth and eighth grades reflect large summer gains 
and smaller than average school year gains. In the third 
grade, however, the high efficiency ratio reflects mod- 



Why Did Summer Bridge Eighth Graders Have Such Large Adjusted Test-Score increases? 

An important finding in this chapter was that adjusted test-score gains were greatest among older students, 
with the largest increases occurring in the eighth grade and the smallest in the third grade. Part of the reason 
for the smaller relative test-score increases among third graders may be the grade equivalent (GE) metric. 
Older students tend to make larger test-score increases than younger students because one more item cor- 
rect on the Iowa Tests of Basic Skills (ITBS) translates into a greater GE increase in the upper grades. When 
we compared the adjusted test-score gains of Summer Bridge students to their learning rates during the 
school year (a method that makes learning gains in Summer Bridge relative to average learning gains at that 
grade level), we found that the learning rate of students who attended Summer Bridge was above what they 
experienced during the regular school year in all three grades. 

The finding that sixth and eighth graders in Summer Bridge experienced large test-score gains does differ 
from previous research, however. Cooper and his colleagues' evaluation of summer school found that reme- 
dial summer programs serving primary grade students have larger program impacts than programs serving 
middle-grade students. Smaller program effects in other studies may reflect program emphasis rather than 
age differences.’ Cooper and his colleagues did find that summer programs in the middle and upper grades 
often focused less on building basic skills and more on general study skills in content areas. Summer Bridge, 
on the other hand, has a mandatory curriculum focused specifically on basic reading and mathematics. This 
suggests that summer programs for low-achieving middle-school students that provide remedial instruction 
in basic skills can produce substantial results. 

A second interpretation is that both incentive and student motivation differ across grades. Chicago's initiative 
relies heavily on the incentive it creates for students to work harder. Eighth graders face the greatest costs in 
not meeting the test-score cutoffs — they don't go to high school. Older students may also have a greater 
capacity to shape their school performance through their own motivation and work effort.^ This would sug- 
gest that Summer Bridge gains among older students are more about the degree to which students are 
motivated rather than about particular curricular foci. 



' Cooper et al, (2000). 

^ Roderick and Engel (2001). 
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est gains in Summer Bridge compared to very small 
school year gains. 

Summer Bridge Test-Score Gains by Students' 
Risk and Demographic Characteristics 

A critical question is whether some students ben- 
efit. more from participation in Summer Bridge than 
others. Not all students who attend Summer Bridge 
face the same task. Some may be very close to the test- 
score cutoffs and need only minimal g^ins. Others face 
significant learning gaps. Wliether programs like Sum- 
mer Bridge provide greater or lesser benefits to stu- 
dents with very large skill gaps is an open question. 
On the one hand, students with the lowest skills may 
benefit most from extended instructional time with 
teachers in more homogenous environments where 
teachers have the opportunity to spend more class time 
on basic concepts. On the other hand, students who 
enter Summer Bridge with very low skills may simply 
be too far behind to catch up in six weeks. This prob- 
lem might be exacerbated by the Summer Bridge cur- 
riculum, which is fast paced, intended to be on ‘'grade 
level,” and is focused on the skills that are covered on 
the ITBS. Students with very low skills may need more 
time, a different level of intervention, and more reme- 
dial content. 

In addition, although the program has a manda- 
tory attendance policy, it also relies on the threat of 
retention to motivate students to work toward meet- 
ing the cutoffs. Students who are closer to the test- 
score cutoffs may be more motivated in the summer 
because they are more likely to believe that the goal is 
within their reach if they work hard. Students who 
have very low skills may feel less able to meet the test- 
score cutoff over the summer and may perceive the 
program more as a punishment than as a support, fur- 
ther exacerbating disengagement. 

We evaluated how Summer Bridge adjusted test- 
score gains varied by students’ prior achievement us- 
ing the risk categories described in Chapter 1. Since 
these risk categories are defined by how far behind the 
ITBS cutoff students were when they entered that 



FIGURES 2-6A AND B 

Learning Gains per Week in Summer Bridge Were 
Substantially Higher Than During the School Year 

Relative Efficiency of Summer to School Year Gains in Reading 



400% -1 
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Relative Efficiency of Summer to School Year Gains in' Mathematics 
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Third grade Sixth grade Eighth grade 



m 1997 m 1998 11 1999 H 2000 

Note: To obtain the summer learning rate, we divided the adjusted. summer gain by 
nine — the number or weeks of summer school plus estimated instructional time at 
the end of the school year after testing. Thus, in 1997, the adjusted test-score 
increase in reading for third graders of 2.5 months converts to a per week gain of 
.28 (calculated by. dividing 2.5 months by 9 weeks). To obtain the school-year 
learning rate, we divided the predicted school -year gain by 37— the number of 
weeks in a typical school year before testing. For third graders who attended 
Summer Bridge in 1997, their average school-year test-score increase was 5.2 
months or a per week gain of .14 (5.2/37). 
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grade, poor performance on the spring ITBS is not 
confounded with estimates of how well a student did 
in Summer Bridge. Students were defined as being at 
high of retention if their prior test-score trends 
indicated they would have to increase their reading/ 
math n*BS score by over 1.5 GEs in one year in order 
to reach the cutoffs. Students were defined as being 
at moderate risk if their prior test-score trends sug- 
gested they would have to increase their ITBS scores 
by between 0.5 and 1.5 GEs. And, students were de- 
fined as being at low risk of retention if their predicted 
prior years test score was 0.5 or less below the cutoff. 
We show results for 1999 only, as we observe similar 
patterns in other Summer Bridge years. 

In the third grade, high-risk students experienced 
the largest adjusted test-score increases in reading (see 
Figure 2-7a). The gains were large in both absolute 
terms and in comparison to their more in ode rate- risk 
peers. In 1999, the average adjusted test-score gain in 
reading among high-risk third graders was nearly four 
months (3.8), compared to smaller (1,5 to 2 months) 
test-score increases for students at low to moderate risk. 
The opposite pattern occurred in mathematics in the 
third grade. Third graders at high to moderate risk of 
retention experienced small adjusted test-score in- 
creases in matherhatics while low-risk students had 
larger increases. 

In the sixth and eighth grades, students who en- 
tered Summer Bridge with test scores closer to the pro- 
motional cutoffs experienced larger adjusted test-score 
increases in the program, with the exception of 
eighth-grade mathematics (see Figures 2-7b and 2- 
7c). For example, in sixth grade, the average Sum- 
mer Bridge student at low risk of retention (students 
whose fifth-grade ITBS reading scores were less than 
0.5 GEs below the promotional cutoffs) experienced 
adjusted test-score increases in Summer Bridge of 
nearly six months (0.57) compared to an average ad- 
justed gain for high- and moderate-risk students of 
slightly over four months. At the same time, these ad- 
justed test-score increases for moderate- and high-risk 
students are still substantial. Thus, high- and moder- 
ate-risk sixth and eighth graders experienced signifi- 
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FIGURE 2-7A 

High-Risk Third Graders Gained the Most in Reading, 
but the Least in Mathematics In Summer Bridge 

Adjusted Summer Bridge Gains by Risk of Retention, 1999 




Ten percent of third graders attending Summer Bridge are in the low-risk 
category, 75% in the moderate-risk category, and 15% in the high-risk 
category. 



FIGURE 2-7B 

Low-Risk Sixth Graders Had Higher Learning Gains in 
Reading and Mathematics 

Adjusted Summer Bridge Gains by Risk of Retention, 1999 
. 8 - 
. 7 “ 




All Low risk Moderate risk High risk 

I Reading S Mathematics 



Eight percent of sixth graders attending Summer Bridge are In the low-risk 
category, 59% in the moderate-risk category, and 32% in the high-risk category. 
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FIGURE 2-7C 

Low-Risk Eighth Graders Had Higher Reading Gains 

Adjusted Summer Bridge Gains by Risk of Retention, 1999 
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these differences were not very large and may have 
occurred by chance. Adjusted test-score gains in Sum- 
mer Bridge did differ by gender. In the third and sixth 
grades, boys had significantly lower adjusted test-score 
gains in mathematics (see Figure 2-9). In the third 
grade, boys experienced smaller test-score gains in read- 
ing as well, even after accounting for differences in 
students’ prior test scores. 



All * Low risk Mode rale risk- High risk ^ . 

H Reading O Mathematics 

This was not statistically significant for reading in regression. Ten percent of eighth 
graders attending Summer Bridge are in the low-risk category, 44% in the 
moderate-risk category, and 46% in the high-risk category. 



ludi Do Summer Bridge Test-Score Increases 
Vary across Schools? 

Another important policy question is whether general 
differences in school quality result in differences in 
summer program quality. The highly prescribed and 
centralized nature of Chicago’s program may reduce 
variation in program quality across schools. Nonethe- 
less, CPS schools continue to have wide flexibility in 
deciding how to staff the program, and teachers have 
flexibility in how they choose to provide instruction. 
Schools that operate efficiently and have good instruc- 
tional programs, better principal leadership, or more 
highly qualified staff may simply be more equipped to 



FIGURE 2-8 

Summer Bridge Gains Do Not Differ Substantially by Student Race 

Adjusted Gains by Race in 1999 



cant test-score gains, although they were 
not as large as those of students closer- 
to the cutoff. 

We also used a hierarchical linear 
model (HLM) to estimate whether stu- 
dents of different racial and ethnic 
groups as well as females and males had 
different outcomes in Summer Bridge 
once we accounted for differences across 
groups in their levels of prior achieve- 
ment and the schools they attended (see 
Appendix D for details on the HLM and 
controls used). After accounting for stu- 
dents’ prior achievement and school 
characteristics, adjusted learning gains 
in Summer Bridge were quite similar 
across racial and ethnic groups (see Fig- 
ure 2-8). African-American third and 
sixth graders did have slighdy lower than 

. . • • * j • L Differences are not statistically significant. Results shown are after controlling for other student and school 

average, learning gains in reading, but characteristics, including school racial composition. 




Third grade Sixth grade Eighth grade 



African-American 
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FIGURE 2-9 

Boys Tend to Have Smaller Summer Bridge Learning Gains 
in Mathematics Than Girls 

Adjusted Gains by Gender in 1 999 




Third grade Sixth grade Eighth grade Third grade Sixth grade Eighth grade 



B Girls S Boys 

Differences are significant In math in third and sixth grades but are not in reading except in third grade. 
Results shown are after controlling for other student and school characteristics. 



FIGURE 2-10 



Students in Schools with Higher School Year Achievement Have Larger 
Test-Score Gains in Summer Bridge 

Adjusted Summer Bridge Gains by School Achievement in 1999 
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Reading 
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Third grade Sixth grade Eghth grade ITiird grade Sixth grade Eighth grade 



Q High-achieving schools 
(over 40% of stu d ents 
at or above national norms 
in reading during the 
school year) 



Moderate-achieving schools 
(25-40% of students at 
or above national norms 
in reading during the 
school year) 



Low-achieving schools 
(less than 25% of students 
at or above national norms 
in reading during the 
school year) 



Results are significant for third grade reading and sixth grade math. Results shown are after controlling for 
student and school characteristics. 



provide efFeccive instruction dur- 
ing the school year and in the 
summer. T hus, a critical question 
is whether summer programs pro- 
duce similar effects across schools 
or whether they exacerbate the 
differences in school quality that 
students experience during the 
school year. 

Even though we do not have a 
measure of school quality, we can 
look at whether test-score in- 
creases in Summer Bridge varied 
by both school-level achievement 
and racial composition. We ob- 
tained these results from the same 
HLM that estimated adjusted 
learning gains across schools while 
taking into account students’ 
achievement levels and demo- 
graphic characteristics. Thus, 
these results can also be inter- 
preted as differences in test-score 
gains across schools if all schools 
served average Summer Bridge 
students. 

. Even after accounting for the 
fact that schools with low achieve- 
ment during the school year 
tended to serve lower achieving 
students during Summer Bridge, 
adjusted learning gains varied sig- 
nificantly across schools. Students 
who attended Summer Bridge in 
high-achieving schools (those 
with higher school year ITBS 
scores) experienced significantly 
larger gains in Summer Bridge 
than students in lower achieving 
schools (see Figure 2-10). These 
differences were more pro- 
nounced in mathematics than in 
reading. Summer Bridge students’ 
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Why Do Third Graders Have Lower Test-Score Improvements? 

The scores of sixth and eighth graders on the Iowa Tests of Basic Skills (ITBS) have improved since Chicago's policy to 
end social promotion was implemented. Third-grade performance increases have consistently lagged behind those for 
sixth and eighth graders, even though they have also Improved since the policy's implementation.’ Given their devel- 
opmental stage, third graders may have less of an ability to understand the threat of retention and, thus, may be 
less motivated than older students to work hard or to take the test more seriously. 



Summer 

Bridge 



Number of students 



GE at chance 



Percent 



Number of students 



3.850 



1.6 



26% 



2,768 



5,287 



2.1 



26.7% 



2,787 



3,755 



1.6 



25.4% 



2,512 



Fewer sixth and eighth graders scored below 
chance, with the exception of sixth graders in 
1997 and 1999. In those years, students took 
the form M of the (TBS, which appears to be 
substantially more difficult than the other 
forms for sixth graders. The number of 
eighth graders scoring below chance has 
declined over time. 

Calculating test-score gains for low-achieving 
students is imprecise for students who score 
below chance. For these students, we cannot 
tell whether the difference between two test 
scores represents an actual increase in stu- 
dent learning or if it is simply the result of 
luckier guessing. A below-chance test score 
does indicate poor reading comprehension, but 
the accuracy of the score is questionable. 



Analysis indicates that the ITBS may not be an adequate tool to measure learning for low-achieving students. Younger 
students in particular may have failed to learn basic test-taking skills. The distribution of raw test scores shows that 
third graders have much higher rates of scoring below "chance" than students in higher grades. Chance is defined as 
the expected score a student would receive if guessing at random. For example, the third-grade reading ITBS has 36 
multiple-choice questions, each with four pos- 
sible answers. A student answering the test 
randomly has a one in four, or 25 percent, 
chance of getting any one of these questions 
correct. Across 36 questions, this results in 
an expected total of nine questions correct (25 
percent of 36). Between 1 1 and 18 percent (de- 
pending on the form) of third graders tested 
answered nine or fewer questions correctly 
(scoring below chance) on the spring ITBS. 

On the summer ITBS, the percentage of third 
graders scoring below chance ranges from 
25 to 27 percent. That is, over a quarter of 
third graders tested after Summer Bridge 
scored lower than they theoretically would 
have by answering "A" to every question. 



Year 


1997 


1998 


1999 I 


2000 


Test Form | 


(M.K) 


(L.K) 


(M,K) 


I (K,L) 


Grade 3 


Spring 


Percent 


11% 


16.7% 


10.5% 


17,8% 



6,355 



1.9 



26.9% 



2,707 



1 \ GE at chance 


1 


1 


1 1-9 


2.1 1 


Grade 6 


Spring 


Percent 


21.1% 


7.2% 


18.9% 


5.4% 




Number of students 


5,765 


2,113 


5,320 


1,435 




GE at chance 


4.6 


3.8 


4.6 


3.9 


Summer 

Bridge 


Percent 


12.9% 


12.3% 


12.9% 


7.8% 


Number of students 


1,118 


993 


922 


579 




1 

GE at chance 


3.9 


3.9 


3.9 


3.8 


Grade 8 


Spring 


Percent 


9.3% 


4.3% 


5.8% 


6.4% 




Number of students 


2,312 


1,171 


1,472 


1,594 




GE at chance 


5.6 


5.0 


5.6 


5.6 


Summer 

Bridge 


Percent 


17.5% 


14.8% 


11.2% 


3.3% 


Number of students 


1,003 


1,037 


656 


326 




GE at chance 


5.6 


5.6 


5.6 


5.0 



Note: "GE at chance" is the expected score a student would receive if 
guessing randomly. 



’ Roderick et al. (1999); Roderick et al. (2000). 
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FIGURE 2-11 

Eighth Graders In African-American Schools Have 
Larger Gains in Summer Bridge 

Adjusted Gains by School Racial Composition in 1999 



. 9-1 
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B African-American schools @ Latino schools H Other schools 

(85% or more African- (85% or more 

American students) Latino students) 

Results are significant for 8th grade reading and mathematics for all three grades. Results shown are after 
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average adjusted mathematics test-score gains were over one month larger 
in high-achieving schools than in the lowest achieving schools in the sys- 
tem'. For third graderS;, these school effects were also found in reading. The 
average estimated summer test-score gain in reading for third graders in 
schools with higher school year performance was nearly three times as large 
as the estimated summer gain for third graders in schools with the lowest 
school year achievement. 

When we accounted for both the characteristics of students and their 
schools, we also found differences in Summer Bridge learning gains by 
the racial composition of the school, particularly in eighth grade (see Fig- 
ure 2-11). Sixth- and eighth-grade students who attended predominantly 
African-American schools experienced larger gains in reading than students 
in predominantly Latino schools. In eighth grade, the adjusted ITBS read- 
ing gains for students in predominantly African-American schools were 
over twice as large as those of students in predominantly Latino schools. 
Previously, we found that test-score gains did not differ by the race of the 
individual student y but here we find a difference in test-score gains based 
on school racial composition. 



Passing and Promotion Rates 

Although improvements in test 
scores are important for measur- 
ing the effectiveness of Summer 
Bridge, ultimately, the purpose of 
the program is to help students 
avoid retention by raising their 
test scores to the promotional cut- 
offs. In the previous chapter we 
saw that students who attended 
Summer Bridge entered the pro- 
gram with significant test-score 
gaps. Were the test-score gains 
that we have identified large 
enough to allow students to meet 
the promotional cutoffs,^ 

Passing and Promotion Rates 
by Grade 

Passing rates at the end ol the 
summer were high among both 
sixth and eighth graders and lower 
for third graders (see Figure 2-12), 
This is consistent with the fact 
that sixth- and eighth-grade stu- 
dents had higher adjusted test- 
score gains in Summer Bridge. 
Between 1997 and 2000, approxi- 
mately half of sixth and eighth 
graders who were required to at- 
tend Summer Bridge met the pro- 
motional cutoffs in both subjects 
by the end of the summer, com- 
pared with 40 percent of third 
graders. Summer Bridge passing 
rates have been relatively consis- 
tent across years, with the excep- 
tion of eighth-grade passing rates 
in the year 2000. As we saw in 
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Chapter 1 and at the beginning 
of this chapter, both school year 
passing rates and adjusted Sum- 
mer Bridge learning gains among 
eighth graders were significantly 
lower in the year 2000. 

In the first several years of the 
policy, some students were pro- 
moted at the end of the summer 
despite not having met the test- 
score cutoffs. While there was no 
official 'waiver” policy, some stu- 
dents were promoted despite not 
meeting the cutoffs if they had ex- 
tenuating circumstances, if their 
principal requested a waiver, or if 
they were very close to the cut- 
offs. In 2000, in response to con- 
tinuing criticism of the use of a 
single test score, as well as civil 
rights complaints, the school sys- 
tem began using multiple mea- 
sures to make promotional 
decisions at the end of the sum- 
mer. These multiple measures in- 
cluded teacher and principal 
recommendations, grades and at- 
tendance in summer school, and 
the use of a range of scores on the 
ITBS. We identified students as 
^‘promoted despite not meeting 
the cutoffs” if they advanced to 
the next grade in September af- 
ter Summer Bridge without 
having met the strict ITBS cut- 
offs in both reading and math- 
ematics. Presumably, these 
students were promoted because 
they met the system s alternative 
criteria for promotion. 



FIGURE 2-12 

Summer Bridge Passing Rates Were Steady, but Promotion 
Rates Increased in 2000 

Passing and Waiver Rates by Grade 




Eighth grade 




T# 



Percent passed 



B Percent waived 



Students who met the cutoff scores in the subjects they needed to pass. Waived students did not meet 
one or both of the cutoffs but were promoted regardless. Only students who remained in CPS the 
following fall are included. 



The move toward expanding the criteria for promotion in the year 2000 
dramatically increased the proportion of Summer Bridge students who were 
promoted. The largest increase occurred in the third grade. In 1999, 49 
percent of Summer Bridge third graders who remained in the school sys- 
tem after the end of the summer were promoted. Most of these students 
met the test-score cutoffs. In 2000, over three-quarters (77 percent) of 
Summer Bridge third graders were promoted, less than half of whom 
met the cutoffs. 



Passing and Promotion Rates by Risk Categories 

Few high-risk students were able to meet the CPS test-score cutoffs. Recall 
that we defined students as high risk if their prior test-score trajectories 
predicted that they would have to increase their ITBS scores by 1.5 GEs or 
more during the promotional gate year. In the previous chapter, we found 
that virtually all high-risk students were required to go to summer school. 
Despite the fact that high-risk students experienced substantial average 
learning gains in Summer Bridge (approximately four months in reading 
for third and sixth graders and six months for eighth graders), very few 
were able to raise their test scores to the cutoffs. Only slightly over 10 
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FIGURE 2-1 3A 

Passing and Promotion Rates for Third Graders in 
Summer Bridge by Risk of Retention 







B Percent passed Percent waived 

percent of Summer Bridge third graders with the 
lowest test scores were able to reach a 2.8 on the 
ITBS in both reading and mathematics (see Figure 
2- 13a). Passing rates were slightly higher among 
high-risk sixth and eighth graders (see Figures 2- 
13b and 2- 13c). Between one-quarter and one-third 
of high-risk Summer Bridge students in these grades 
met the cutoffs in reading and math at the end of 
the school year. 

Passing rates among moderate-risk students were 
also lowest in the third grade. Only about 40 percent 
of moderate-risk third graders compared with almost 
60 percent of sixth and eighth graders who were at 
moderate risk were able to raise their test scores to the 
cutoffs by the end of summer. 

The use of the expanded promotional criteria dra- 
matically increased the number of high- and moder- 
ate-risk students who were promoted at the end of 
Summer Bridge. Between 1997 and 2000, the pro- 
portion of moderate- risk third graders who were pro- 
moted at the end of the summer increased from 62 to 
79 percent and increased among high-risk third grad- 
ers from 32 to 50 percent. Across all three grades, low- 
risk students in Summer Bridge had very high passing 




Passing and Promotion Rates for Sixth Graders in 
Summer Bridge by Risk of Retention 




B Percent passed ^ Percent waived 



rates and, after the expanded promotional criteria were 
applied in 2000, their chance of being retained was 
extremely low. 

The purpose of examining passing and promotion 
rates was to answer the question of whether Summer 
Bridge participation paid off for students. If we look 
only at whether students met the test-score cutoffs, 
results for high- and moderate-risk students are mixed. 
Very few high-risk students, despite test-score increases, 
were able to raise their scores to the cutoffs. For stu- 
dents at moderate risk, proportions meeting the cut- 
offs in 1997 ranged from a low of approximately 40 
percent in the third grade to close to 60 percent in the 
sixth and eighth grades. The expanded promotional 
criteria had a large impact on raising promotion rates 
for high- and mode rate- risk students. The proportion 
of students who succeeded in meeting the cutoffs com- 
bined with those students who were promoted using 
expanded promotional criteria meant that the major- 
ity of Summer Bridge students were promoted at the 
end of the summer. In the year 2000, about 80 per- 
cent of mode rate- risk third and sixth graders and 86 
percent of moderate-risk eighth graders were promoted 
at the end of the summer. 
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FIGURE 2-13C 

Passing and Promotion Rates for Eighth Graders in 
Summer Bridge by Risk of Retention 
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What Have We Learned and How Do We 
interpret These Results? 

The goal of this chapter was to assess two sets of out- 
comes in Summer Bridge. First, to what extent did 
students have test-scoregains in Summer Bridge? And, 
second, to what extent did participation in Summer 
Bridge allow students to be promoted to the next grade? 

Taken together, our results provide strong evidence 
that Summer Bridge had positive short-term impacts 
on test scores in all grades in both reading and math- 
ematics. Sixtli- and eighth-grade students experienced 
significant test-score increases in the program. In all 
three grades, students who attended Summer Bridge 
experienced a higher rate of learning during the sum- 
mer than during the regular school year. And, Sum- 
mer Bridge produced test-score gains for students 
across a variety of demographic characteristics. There 
were some differences in the size of test-score gains by 
student achievement and gender. Sixth- and eighth- 
grade students who entered Summer Bridge with test 
scores closer to the cutofiFs tended to have larger test- 
score increases than students with persistently lower 
test scores. Also, boys tended to have smaller test-score 
gains than girls. These differences, however, were not 



large. Taken together, the lack of any consistent dif- 
ferences in program impact by student characteristics 
may be one of the most positive findings in this chap- 
ter because it suggests that, despite concerns about run- 
ning large-scale summer programs, Summer Bridge 
produced rather uniform benefits across students. 

Results for third grade are less positive. Summer 
Bridge program results in the third grade looked quite 
similar to those of other programs but were relatively 
inconsistent across years. Third graders with the low- 
est sldlls had the greatest gains in reading in that the 
gains were substantially above those of moderate- and 
low- risk students. Thus, for this small but important 
group of high-risk third graders (15 percent of Sum- 
mer Bridge students), it appears that Summer Bridge 
was effective in raising their reading test scores. Ef- 
fects for third graders at moderate to no risk of reten- 
tion were much more modest. 

Although we do not find evidence that there are 
significant and important differences in program ef- 
fects across students, the schools that students attended 
mattered. Summer Bridge adjusted test-score gains 
were significantly greater in schools where, on aver- 
age, the entire student body is higher performing. This 
occurred even after we accounted for differences in 
prior achievement of individual students, differences 
in passing rates during the school year, and differences 
in Summer Bridge program size. Thus, even for their 
lowest performing students, high-achieving schools 
were able to provide more effective learning environ- 
ments in Summer Bridge, 

The differences by school achievement level were 
most pronounced in the third grade and in mathemat- 
ics. Third-grade reading teachers have the difficult task 
of teaching reading comprehension to students who 
may not have developed basic reading skills yet. In 
addition, teaching mathematics, particularly in the 
upper grades, requires that teachers have sufficient con- 
tent knowledge. Thus, while the highly structured na- 
ture of Summer Bridge may produce more uniformity 
in content coverage, it does not ameliorate overall 
differences in school quality. A caveat to this conclu- 
sion is that adjusting test-score gains for differences in 
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participants’ prior test scores and demographic back- 
grounds may not capture all of the differences in these 
schools’ student populations. Lower performing CPS 
schools may serve students with greater barriers to 
learning because, among other reasons, they live in 
neighborhoods where students’ summer school par- 
ticipation and performance may be delete riously af- 
fected (for example, by safety issues). Because we have 
not accounted for differences in students’ actual at- 
tendance in Summer Bridge, neighborhood effects, or 
differences in home support for learning over the sum- 
mer, we may not be adequately capturing differences 
in the student populations served by high- and low- 
achieving schools during the summer. 

Summer Bridge effects declined significantly in 
mathematics in 2000 for all grades and in both read- 
ing and mathematics for eighth grade. At this point, 
we lack the data to determine whether this reflects a 
trend or a one-time cohort effect. Three policy changes 
may have affected the 2000 results. First, cutoff scores 
were increased in sixth and eighth grades and, as a 
result, many more students were required to attend 
Summer Bridge. Many of these students were very close 
to the cutoffs and had to gain very litde, which may 
have decreased their motivation. Second, the system 
began to shift, as we saw, toward expanded promo- 
tional criteria. Eighth graders may have been less mo- 



tivated to work hard if they were expecting to be pro- 
moted after Summer Bridge. And third, across all 
grades. Summer Bridge teachers were told that they 
could shift instructional time away from mathematics 
and toward reading because so many more students 
were at risk in reading. This may have had a negative 
effect on test-score gains in mathematics in 2000. In 
any case, whether this was a one-time effect or repre- 
sents a new trend remains to be seen. 

Finally, this chapter highlights the central tension 
of creating effective policies under high-staltes testing. 
On the one hand, findings in this chapter suggest that 
using single test-score cutoffs for promotional deci- 
sions may set up for failure those students with the 
lowest skills. Despite test-score gains in Summer Bridge 
that were larger than average for these students, very 
few high-risk students were able to raise their test scores 
to the promotional cutoffs. The expansion of the pro- 
motional criteria means that students with the lowest 
skills had a significantly higher likelihood of being pro- 
moted to the next grade in 2000, which could be in- 
terpreted as a means of rewarding work effort and an 
acknowledgement of learning gains despite not hav- 
ing met the cutoffs. On the other hand, one could 
argue that the system is no longer working toward end- 
ing social promotion and that many students are now 
being promoted without.adequate skills. 
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Students' Experiences 



S tudent test scores and promotion rates hold the most direct evidence 
of whether the CPS Summer Bridge program met its objectives. They 
may not, however, be the best measure of whether the program pro- 
vided students with positive learning experiences. For example, one might 
argue that students increased their test scores over the summer simply be- 
cause the curriculum covered the skills they needed to score higher on the 
ITBS. In addition, students may be more likely to work harder on the test 
and pay attention when it matters most: when it is their last chance to 
avoid retention. Viewed this way, tlie Summer Bridge test-score gains we 
observed in the previous cliapter would be largely the result of “testing” and 
“test preparation” effeas rather than of a positive academic environment. 

At the same time, prior research suggests that the critical programmatic 
components of Summer Bridge may lead to both achievement gains and a 
positive learning environment. There is evidence that small class sizes pro- 
duce benefits during the school year and the summer, particularly for low- 
performing students.^ The combination of focused instruction, small class 
size, and motivated students may provide a more intensive and personal- 
ized learning environment. Research on effective schools has consistently 
found that students learn more and are more successful when their class- 
rooms combine high levels of personal support, often referred to as per- 
sonalism, and strong expectations for students to work hard, or academic 
press. ^ The programmatic characteristics of Summer Bridge might have 
produced both greater academic press and personalism for students. We 
would not expect students to report particularly positively on the academic 
climate or their relationships with teachers in Summer Bridge if their learn- 
ing gains were simply a result of working hard on the test itself or engaging 
in specific test prep activities. 



_ BEST COPY AVAILABLE 

54 



52 



CHARTING REFORM IN CHICAGO 



This chapter draws on spring and summer survey 
results and student interviews in 1999 to examine stu- 
dent reports of their experiences in Summer Bridge. 
We focus on three questions: How did students de- 
scribe their experiences in Summer Bridge classrooms? 
Wliat is the difference between students' portrayal of 
the instructional environment during the school year 
and during Summer Bridge? And, are there differences 
across schools and students in descriptions of the pro- 
gram — for example, did students with very low skills 
report different experiences in Summer Bridge than 
students closer to the test-score cutoffs? 



Student Experiences in Summer Bridge: 
Survey Results 

In 1 999, the Consortium on Chicago School Research 
conducted a survey of sixth- and eighth-grade Sum- 
mer Bridge students. The survey included a set of ques- 
tions that asked students to compare their experience 
in Summer Bridge to the school year and also asked 
students to respond to general questions about the 
program. Because many students attending Summer 
Bridge have low reading scores, the survey, when pos- 
sible, used simple response categories, such as “a lot,” 
“a little,” and “no.” 



This chapter uses three sources 
of data on Chicago Public School 
(CPS) students: (1) surveys of 
sixth- and eighth-grade students 
who attended Summer Bridge in 
1 999 and all of their teachers, (2) 
school year surveys conducted in 
spring 1999, and (3) interviews 
with 48 students who attended 
Summer Bridge and were part of 
a longitudinal study of Chicago's 
efforts to end social promotion. 

Survey Samples. In spring 1999, 
the Consortium on Chicago 
School Research conducted its 
biannual survey of sixth-, sev- 
enth-, and eighth-grade elemen- 
tary school students and ninth- 
and tenth-grade high school stu- 
dents, and all teachers and prin- 
cipals in the system. This is a 
regularly scheduled survey that 
was also administered in 1994, 
1997, and 2001 . During summer 



Data Used in This Chapter 

1999, the Consortium also adminis- 
tered surveys to sixth- and eighth- 
grade Summer Bridge students and 
teachers in the program. Summer 
surveys were administered during the 
last week of the program in August. 
The summer survey asked students 
questions about their experiences in 
Summer Bridge, their attitudes to- 
ward testing, and their perceptions of 
support from teachers and parents. 
It also repeated two series of items 
from the spring survey that were 
meant to assess the academic orien- 
tation of classrooms and the level of 
personal support students received 
from teachers (see Appendix E, Meth- 
odology for Developing Survey Mea- 
sures). These two sets of items are 
core indicators developed by the Con- 
sortium and have been linked over 
time to student and school perfor- 
mance. Repeating the measures al- 
lows us to compare students' 
responses over time. In 1 999, 56 per- 



cent of sixth- and eighth-grade 
students responded to the survey 
during the school year. 

Sample Size. In 1 999, 7,265 sixth- 
grade students and 6,128 eighth- 
grade students were required to 
attend Summer Bridge and were 
tested at the end of the summer. 
Appendix G describes the demo- 
graphic and achievement compo- 
sition of students who attended 
Summer Bridge in 1999, students 
who completed the summer sur- 
veys, and students who completed 
both summer and school year sur- 
veys. Surveys of students in Sum- 
mer Bridge were collected from 88 
percent of schools that had sixth- 
and eighth-grade classrooms. 
Completing the summer surveys 
were 4,829 sixth graders and 4,225 
eighth graders, resulting in a sur- 
vey completion rate of 66 percent 
of sixth and 69 percent of eighth 
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If given the choice, most students would not go to 
summer school. Not surprisingly, many students re- 
ported that they were “mad” that they had to attend 
Summer Bridge. Over three-quarters of sixth graders 
and 88 percent of eighth graders were mad that they 
had to go (see Figures 3-la and 3-lb). Despite this, 
students portrayed Summer Bridge as a positive learn- 
ing environment in which they were expected to work 
hard and where teachers supported their efforts. 
Eighty-one percent of sixth graders and 84 percent of 
eighth graders reported that they worked harder in 
summer school than the regular school year. Students 
also reported that an important distinction between 



the school year and summer was the amount of atten- 
tion they received from teachers. Over 85 percent of 
sixth and eighth graders reported that teachers had 
more time to help them in Summer Bridge than dur- 
ing the regular school year. The fact that students felt 
both that they were working harder and that they were 
getting more support from teachers likely contributed 
to students’ perceptions that they were learning. About 
three-quarters of sixth and eighth graders reported 
learning more in Summer Bridge than during the 
school year. 

The portrayal of Summer Bridge classrooms as 
being more conducive to supporting students’ work 



graders who took the Iowa Tests of 
Basic Skills (ITBS) in August. 

Of the schools that participated in 
the summer survey, over 80 per- 
cent also participated in the spring 
survey. In these schools, 2,519 
sixth graders and 2,303 eighth 
graders completed the spring sur- 
vey. Thus, we were able to obtain a 
matched sample for 55 percent of 
the eighth graders and 52 percent 
of the sixth graders who completed 
summer surveys. The survey and 
matched sample underrepresents 
African-American students, largely 
because predominantly African- 
American schools were less likely 
to complete surveys during both 
the school year and summer. 

The Qualitative Study. In 1999, as 
part of the evaluation of Chicago's 
initiative to end social promotion, 
the Consortium began a longitudi- 



nal study of 102 low-achieving Afri- 
can-American and Latino sixth and 
eighth graders from five CPS elemen- 
tary schools. The study followed stu- 
dents, their families, and their 
teachers in preparing for and taking 
the ITBS over the summer and in the 
retained or promoted years. The 
baseline interview was conducted in 
the spring prior to testing. Students 
were then interviewed immediately af- 
ter taking the ITBS and once during 
the summer. Retained students were 
interviewed twice during their re- 
tained year and those who were pro- 
moted were interviewed once. 

The qualitative study focused on the 
experiences of students who were at 
risk of being retained and who at- 
tended schools in two of the five 
neighborhoods with the highest re- 
tention rates, those neighborhoods 
where one-third or more of included 
students were being retained. One of 



the two neighborhoods selected for 
the study was predominantly Afri- 
can-American and the other was 
mixed Latino and African-Ameri- 
can. Within each neighborhood, we 
identified two schools for partici- 
pation. The fifth school was a pre- 
dominantly Latino school with a 
large bilingual education program. 
This school was chosen in order 
to examine how Latino immigrant 
families and students in bilingual 
education were affected by the 
policy. 

Within each school, approximately 
ten low-achieving sixth graders and 
ten low-achieving eighth graders 
were selected and recruited for the 
study. At the end of the school year, 
39 (38 percent) of the 102 students 
met the test-score cutoffs and were 
promoted. There were 63 who 
needed to go to Summer Bridge, 
and of those we interviewed 48. 
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FIGURES 3-1AANDB 

Summer Bridge Students Are Very Positive about 
Their Summer School Experience 

Sixth Graders 



Compared to regular school year 

I 

Wor1<ed harder in summer school 
My teacher had more time to help 
Learned more in summer school 
Liked summer school better 
Classmates helped me more 

In general . . . 

I was mad I had to go 
Reading was too hard 
Math was too hard 
Summer school was boring 
Summer school was fun 
I hated summer school 
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Eighth Graders 



Compared to regular school year . . . 



Worked harder in summer school 
My teacher had more time to help 
Learned more in summer school 
Liked summer school better 
Classmates helped me more 

In general . , . 

I was mad I had to go 
Reading was too hard 
Math was too hard 
Summer school was boring 
Summer school was fun 
I hated summer school 
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Results are from the 1999 Summer Bridge survey. 



effort than classrooms in the regu- 
lar school year was also reflected 
in students' overall assessment of 
their experiences. Over 70 per- 
cent of sixth and eighth graders 
reported liking summer school 
better than school in the regular 
year. And, while students reported 
that they worked hard in summer 
school, only about a third felt that 
the work was too difficult. 

These survey responses are cer- 
tainly positive. They depict Sum- 
mer Bridge as an environment 
where studen ts work hard and feel 
supported by teachers. Student 
reports suggest that Summer 
Bridge may provide an atmo- 
sphere that is conducive to learn- 
ing for low-achieving students. 
Students were also positive about 
Summer Bridge in their inter- 
views. We need to be careful, 
however, in interpreting the re- 
sults from cross-sectional surveys 
because they offer us no frame 
of reference or real opportunity 
to compare student responses 
over time and in different envi- 
ronments. Also, we are relying 
on students' memory of the 
school year for these questions, 
which may not always be highly 
reliable or accurate. 

We can examine the extent to 
which student experiences in 
Summer Bridge differed from the 
school year by looking at changes 
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between the school year and Summer Bridge on two 
critical indices of classroom environments, personal- 
ism and academic press. We were able to match the 
Summer Bridge surveys of 55 percent of eighth grad- 
ers and 52 percent of sixth! graders to surveys they com- 
pleted earlier in 1999. We then created the two 
measures, academic press and personalism, with the 
Summer Bridge survey items using Rasch analysis. 
These measures had already been created using the 
same items from the school year survey. For each mea- 
sure, the questions were combined into a single scale 
that ranged from 0 to 10 and placed on a common 
metric in order to allow for comparison between the 
regular school year and Summer Bridge.^ 

Differences in School Year and Summer Bridge 
Reports on Academic Press and Personalism 

Students who attend Summer Bridge are those CPS 
students with the lowest school year achievement. On 
the spring surveys, these students were more negative 
than average sixth and eighth graders in their reports 
of academic press and personalism. For example, the 
median score on the measure of academic press among 
sixth graders who met the test-score cutoffs during the 
school year was 4.8 (see Figure 3 -2a). On the same 
survey, the median score for academic press among 
sixth graders who later attended Summer Bridge was 
4.4. Three months later, when the same students were 
asked about academic press in Summer Bridge, they 
were much more positive, with a median score of 6.0. 
Indeed, during the summer, well over 75 percent of 
sixth-grade Summer Bridge students had higher scores 
on academic press than the median sixth grader dur- 
ing the school year. We see the same pattern of results 
in eighth grade, and in measures of personalism in both 
grades (see Figures 3-2b, 3-3a, and 3-3b). In each case, 
students who attended Summer Bridge were more 
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FIGURES 3-2 A AND B 

During Summer Bridge, Students Report Much Higher 
Academic Press Than during the School Year 

Sixth Graders' Reports of Academic Press 
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Is Summer Bridge Really Different? A Qualitative Look 



In 1999, we conducted intensive interviews with 48 
sixth- and eighth-grade students who attended Sum- 
mer Bridge.’ These students were part of a longitudinal 
study of over 100 low-achieving students that was con- 
ducted as part of a larger evaluation of the initiative to 
end social promotion. Interviews focused specifically 
on exploring students' experiences in summer school 
to elicit comparative information between school year 
and Summer Bridge experiences. 

Most Siuderas Fell Summer Bridge Was a Good Idea 
and That They Had a Good Eiperience 

When asked whether Summer Bridge was a good idea 
or whether they liked Summer Bridge, 60 percent of the 
students interviewed responded positively. Only three 
students (6 percent) reported an extreme dislike of the 
program. The second chance it afforded and the sup- 
port it gave students to raise their test scores were pri- 
mary reasons given by students for why the program 
was a good idea. As one student explained, "Yes, it's a 
good idea because they give you an opportunity to take 
the exam and they prepare you well." When asked if she 
liked the program, this student responded, "Yes, because 
it’s fun to come, and also because they teach us good 
things." The focus on the Iowa Tests of Basic Skills 
(ITBS) was critical for many and was linked to feelings 
that teachers cared. For example, an African-American 
eighth grader explained that he felt his Summer Bridge 
teachers were helping him more than his teachers dur- 
ing the regular school year: "The teachers in the regular 
school year, they had us get ready for the [ITBS] at the 
last minute; they didn't help us right away, and in sum- 
mer school they helped us right away." 

For many students, appreciating the second chance and 
learning were connected. A Latino eighth grader ex- 
plained that he thought summer school was a good idea 
because, "[At] some other school they won't give you 
another chance. And they give you a chance in summer 
school so, you know, I basically wanted to go." When 
asked what was the best thing about the program, how- 
ever, Fernando responded "learning." Other students 
also expressed that they noticed their skills were im- 
proving more generally. One African-American student 



who needed substantial improvement in reading to pass 
the test, felt after the summer that he was prepared, 
"because studying for this test next week — I feel my 
reading's going faster." Another African-American eighth 
grader was initially disappointed by having to go to 
summer school and felt it was a bad idea. Yet, when 
she was interviewed during Summer Bridge, She com- 
mented that she had changed her mind: "Because now 
I see that I learned a lot of things that I didn't know, and 
it’s good for me; it's for my benefit so I could have a 
second chance to go to the ninth grade." 

Summer Bridge Learning Erwirooment Oierent 
from ie School Year 

Almost 88 percent of students interviewed (42 out of 
49) described their Summer Bridge classrooms and re- 
lationships with teachers as different from the school 
year, and 26 out of 48 described their classroom envi- 
ronment in Summer Bridge as substantially better than 
during the school year. When students talked about what 
was different, they focused primarily on two aspects of 
Summer Bridge: teachers slowed the pace and they pro- 
vided more individual attention. Students also felt that 
teachers made sure everyone understood the lesson. 
For example, when a Latino eighth-grader was asked 
what Summer Bridge teachers did that regular school 
year teachers didn't, he replied: "Explain the work real 
carefully; and if you have an answer or problem, you 
tell them and they do it all over again for you so you 
can understand it. That’s way better — they don't hardly 
do that in the regular school year. They explained it real 
fast, that's it." Another student had a similar response, 
"This summer, I get a better understanding. Because 
Mr. B, he breaks stuff down for us. Nope, he won't stop 
'till everybody's doing it.... He makes sure you under- 
stand. If you say you understand and you really don't, 
he'll start asking you questions about the story." 

Indeed, the attention to explanation was viewed by stu- 
dents as a unique characteristic of instruction in Sum- 
mer Bridge. Students experienced a concern for their 
own understanding and an interaction with teachers 
around problems that they did not feel they usually re- 
ceived. As one eighth grader explained, he felt he was 
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ready for the ITBS "because he really explains it to us ... . 
If we don't understand, just keep asking, keep asking; 
he'll show us how." And another student commented 
that his teacher was helping him a lot because "She gives 
us a better understanding. If we don't know it, she ex- 
plains it to us. And she'll do one problem for us; she'll 
let us try to do it, by ourselves." 

Part of students' perceptions that teachers in Summer 
Bridge were more concerned about their understanding 
pointed to differences in the level at which classes were 
taught. It also seemed to be driven by students' percep- 
tions that they received more personal attention in the 
program. When asked if his Summer Bridge teacher was 
helping him, an African-American eighth grader re- 
marked, "If I need help, I go to her desk and ask her. 
She'll, like, explain it one by one — she'll tell me to read 
this paragraph and then read that question and then look 
at my decision." An African-American sixth grader had 
a similar reaction, "Sometimes he'll take us by ourselves 
and then he teaches us; and, like, teachers they don't do 
that." When asked if he got that attention during the regu- 
lar school year, he responded, "Not all the time." An- 
other student explained; 



It's much more boring in regular school. Like, teach- 
ers should be like Ms. D. She teaches us, and when 
we do something wrong she comes and helps us. 
But see, regular teachers don't do that. Summer 
school teachers, they like tell you, "Okay. This is 
wrong," and they fix it for you; and then regular 
teachers, they're like, "This is wrong, try it again." 
But the summer school teachers explain it, and usu- 
ally in regular school they don't. 

Students who reported that they didn't like Summer 
Bridge, although a small group, often expressed frus- 
tration with the slow and repetitive pace. For example, 
in one classroom, a student praised the teacher for go- 
ing slow and paying attention to students' understand- 
ing. For another student in the same classroom, this 
slow pace was frustrating. She explained, "It's very bor- 
ing, and the teachers that we got, it's like the same stuff 
we did before.... It's a continuation... .And if we men- 
tion [a topic] to [the teacher], he'll spend the whole. . .day 
on that." 



' This analysis was taken from a paper written by Susan Stone and Mimi Engel that was presented at the 2001 annual meeting of the 
American Educational Research Association in New Orleans, LA. 




negative about their teachers than the average CPS 
student during the school year, but were substan- 
tially more positive during the summer. These 
marked changes in both the median and the distri- 
bution of student responses suggest that Summer 
Bridge provided a very different learning environ- 
ment for these students. 

Wliicii Students iftere Mere Pj)si®e? 

Student interview responses indicate that students liked 
Summer Bridge because they appreciated the second 
chance it gave them and felt that they received a great 
deal of support and attention from teachers to help 



them raise their ITBS scores. We might expect, how- 
ever, that some did not experience Summer Bridge 
so positively. Students with very low skills, for ex- 
ample, might feel that the goal of meeting the test- 
score cutoffs is out of reach. This could lead them 
to view Summer Bridge as a punishment, making the 
program a less positive learning environment. 

In order to investigate differences across students 
in their assessments of the program, we used a hierar- 
chical linear model described in detail in Appendix D 
to examine test-score gains. Here, we looked at differ- 
ences across students in changes in academic press and 
personalism, as well as differences in their overall as- 
sessments of Summer Bridge (see Appendix F). The 



BEST COPY AVAILABLE 



60 



58 



CHARTING REFORM IN CHICAGO 



FIGURES 3-3AANDB 

During Summer Bridge, Students Report Much Higher 
Personalism Than during the School Year 

Sixth Graders' Reports of Personalism 
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Questions That Were Asked of Students 
in the School Year and Summer 

Academic Press 

To what extent do you (strongly disagree, dis- 
agree, agree, strongly agree) that your teacher 
[or summer school teacher] . . , 

Encourages me to do extra work when I 
don't understand something. 

Praises my efforts when I work hard. 

Cares if I don't do my work in class. 

Expects me to do my best all the time. 

Expects me to do my homework 
every night. 

Thinks it is very important that I do well. 

Cares if I get bad scores on the Iowa (ITBS) 
[summer only]. 

Personalism 

To what extent do you (strongly disagree, dis- 
agree, agree, strongly agree) that your teacher 
[or summer school teacher] , . . 

Really listens to what I have to say. 

Helps me catch up if I am behind. 

Notices if I have trouble learning 
something. 

Is willing to give extra help on schoolwork 
if I need it. 

Believes I can do well in school. 
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measure “overall assessment of Summer Bridge” com- 
bines students’ answers to questions about their Sum- 
mer Bridge experiences (see Figures 3- la and 3-1 b on 
page 54) into a single summary indicator."* This analy- 
sis examines how changes in academic press and per- 
sonalism, as well as scores on the overall assessment of 
Summer Bridge, varied by students’ prior achievement, 
demographic characteristics (race and gender), by the 
race and achievement levels of their schools, and Sum- 
mer Bridge program characteristics (class size and the 
percentage of students the teacher Icnew before Sum- 
mer Bridge). 

In the previous chapter, we found that students with 
the highest risk of being retained had slightly smaller 
adjusted test-score gains, on average, in Summer 
Bridge, and that students in very low-achieving 
schools also had smaller test-score gains. Students 
at high, moderate, or low risk of retention, how- 
ever, did not differ substantially in their assessment 
of summer school. There was little difference across 
all three measures derived from student survey re- 
sponses by school-level achievement or by their prior 
test scores. Eighth graders at high and moderate risk 
of retention had less change in their perception of per- 
sonal support from teachers than students at low risk. 
These results were moderate in size, however, and were 
not reflected in other indicators. 

A consistent pattern in this analysis was that males 
were more negative about the personal environment 
of Summer Bridge. . Boys in both the sixth and eigh th 
grades were more negative in their overall assessment 
of the program and showed less of an increase in their 
perceptions that teachers showed them personal sup- 
port.^ In the previous chapter, we also found that boys 
had lower test-score gains in mathematics. Thus, boys 
do appear to be having somewhat different experiences 
in the program. One interpretation of this pattern is 
that boys are less likely to be motivated by the threat 
of retention and less likely to view the focused envi- 
ronment as a positive experience. For example, it is 
possible that girls, on average, perceive increased at- 



tention from teachers differently than boys. Boys may 
be less likely to perceive pressure from teachers as sup- 
portive. An alternative explanation is that teachers are 
actually treating boys and girls differently and that girls 
are getting more support than boys. 

In the previous chapter, we found that students in 
predominantly African-American schools had larger 
test-score gains, on average, than students in predomi- 
nantly Latino, mixed race, and integrated schools. We 
also found that eighth graders in predominantly Afri- 
can-American schools had larger increases on the mea- 
sure of personal attention from teachers than those in 
predominantly Latino schools, and that eighth grad- 
ers in African-American schools perceived Summer 
Bridge more positively (see Figures 3-4 and 3-5). We 
do not know whether the differences in students’ ex- 
periences across schools reflect differences in their mo- 
tivational responses or differences in teacher behavior 
and classroom environments. 



FIGURE 3-4 

Eighth Graders in African-American Schools Show a 
Larger Increase in Personalism in Summer Bridge 
than Eighth Graders in Other Schools 
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Results shown are from the 1999 school-^^ar and Summer 
Bridge surveys after controlling for student and school 
characteristics. Results for sixth graders are similar. 
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FIGURE 3-5 

Eighth Graders in African-American Schools Are 
More Positive about Summer Bridge than Those in 
Latino Schools 
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Results are from the 1999 Summer Bridge survey. Results for 
sixth graders are similar. 



What Have We Learned? 

In many ways, these findings about Summer Bridge 
students should not be surprising. First, small class size 
provides teachers with more opportunities to pay at- 
tention to individual students. The fact that students 
and teachers are working toward a common goal with 
a focused set of instructional objectives may also have 
led students to feel they were receiving more attention 
during Summer Bridge than the regular school year. 
Research on motivation has found that classrooms 
where teachers and students are working under the 
pressure of external assessments have lower levels of 
anti-academic peer norms and increased peer support 
for achievement when compared to control class- 



rooms.^ There is some evidence that low-achieving stu- 
dents respond positively when an evaluation is based 
on predetermined scores that focus on self-improve- 
ment and on which progress is measurable and can be 
charted over time.' T hus, one hypothesises that stu- 
dents were positive about Summer Bridge because it 
provided them more focused academic support in a 
more personalized environment around a goal that they 
cared about. 

A second hypothesis is that Summer Bridge is a more 
effective learning environment for students not because 
of the extra support it provides, but because the pace 
and structure of the program better meet the learning 
needs of low-achieving students. Our qualitative in- 
terviews suggest that low-achieving students felt more 
supported in Summer Bridge because the pace and 
level of instruction were geared to their skills rather 
than to those of their higher achieving peers. In addi- 
tion, Summer Bridge may also provide a more focused 
instructional environment leading to higher levels of 
academic press. Since students were attending Sum- 
mer Bridge for only three hours per day and receiving 
instruction in only two subjects, they may have felt it 
was more fast paced and focused, leading to the im- 
pression that they worked harder in the summer with 
less down time and interruptions. This suggests that 
students were working harder and felt like they were 
learning more in Summer Bridge because teachers may 
have provided them the support, structure, and chal- 
lenge they lacked during the school year. Thus, it is 
possible that students were working harder and were 
getting more attention in Summer Bridge because 
teachers were providing more focused instruction and 
keeping students on task. In the next two chapters, we 
focus on these issues. 
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n many ways, teachers in the CPS Summer Bridge program face a daunt- 
ing task. They are asked to talce a group of students who have just failed 
to meet the test-score cutoffs and find a way to raise their scores for 
promotion in only six weeks. Nonetheless, the program does provide teach- 
ers two advantages over the school year: a highly structured curriculum 
and small class size. This chapter looks at Summer Bridge teacher surveys 
conducted in 1999 and teacher interviews conducted during the program 
in 2000 to address four main questions. First, how did teachers assess the 
quality and usefulness of the Summer Bridge curriculum? Second, how 
did teachers attempt to meet their students’ needs? Third, were there ele- 
ments of the program or characteristics of the teachers themselves that 
influenced teachers’ practice and perceptions? And, finally, is there evi- 
dence that teacher characteristics and practices and school characteristics 
are associated with students’ test-score gains in Summer Bridge? 

Teacher Assessments of the Summer Bridge Curriculum 

The prescribed Summer Bridge curriculum can be viewed as either a sup- 
port or a hindrance. It may provide a much needed break for teachers who, 
after teaching all year, do not need to design curricula or lesson plans for 
summer. The lesson plans and materials cert^nly make teaching in the 
program easier and provide guidance for raising students’ scores. On the 
other hand, a mandator)'^ curriculum could be perceived as a limitation by 
teachers if they do not believe that it will meet students’ instructional needs 
or if they dislike its structure or content. 
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Teachers were asked about three aspects of the Summer Bridge curricu- 
lum on the 1999 surveys: the appropriateness of the content and quality 
of the instructional materials, the match between the materials and stu- 
dent ability, and the pacing of the lesson plans. 



Approprialeness of Cotifeiil and Qoaiy of iateriais 

Teachers were very positive about the content of the Summer Bridge cur- 
riculum and the quality of its instructional materials. Eighty percent of 
teachers surveyed reported being somewhat to very satisfied with the con- 
tent of the lesson plan guides (see Figure 4-1). Eighty-six percent of teach- 
ers were somewhat to very satisfied with the quality of the materials. When 
asked about the reading and mathematics curricula, most teachers agreed 
that the materials were “useful and appropriate” and that the curriculum 
was “well organized and of high quality” (see Figure 4-2). Teachers were 
more positive about the mathematics curriculum than reading. 

FIGURE 4-1 

Summer Bridge Teachers Were Satisfied with the 
Lesson Plan Guides and Instructional Materials 



Although they were positive 
about the quality of the instruc- 
tional materials, teachers were less 
inclined to report that the mate- 
rials were useful for meeting their 
students’ needs. For example, 
nearly half of teachers stated that 
they were very satisfied with the 
quality of the materials, but only 
12 percent strongly agreed that 
“the reading materials were use- 
fiil and appropriate.” These dif- 
ferences suggest that although 
teachers may have been positive 
about the overall focus of the 
curriculum, they were less sure 
about whether the materials 
were at the appropriate level for 
their students. 



Overall, how satisfied are you with the 
content of the lesson plan guides? 

How satisfied are you with the 
quality of the instruction^ materials? 



H Very Somewhat 

satisfied satisfied 




20 40 60 80 100 

Percent of teachers 



B Somewhat | Very Undecided 

dissatisfied dissatisfied 



Results are from the 1999 Summer Bridge teacher survey. 



iatco to Sludems' Skiils 

We looked at teachers’ assess- 
ments of the match between the 
materials and students’ needs by 
asking teachers if they felt that the 
curriculum was too difficult or 



Data Used in this Chapter 

This chapter draws on two primary sources of data: surveys of Summer Bridge teachers conducted in 1999 and 
in-depth interviews conducted with teachers during Summer Bridge in 2000. The interviews were part of our 
larger qualitative study of Summer Bridge in which we observed classrooms in 12 schools (roughly one teacher 
in each of the third, sixth, and eighth grades) over the course of the summer. At the end of the program, teachers 
who were observed were interviewed, resulting in a total of 37 complete interviews (four teachers declined to be 
interviewed). Interviews lasted approximately 45 minutes to an hour. The interview protocol included questions 
in the following three areas: first, information about students — number of students, absences, tardiness, stu- 
dents' level of risk, students' academic needs and how the teacher chose to address those needs, and student 
motivation; second, questions about the curriculum — whether it was at the right level for students, quality of 
both reading and mathematics lessons and materials, pacing, and whether teachers felt they had time to work 
individually with students: and third, whether teachers felt they had adequate support to teach in Summer Bridge — 
questions about monitors, whether teachers had tutors/aides, and whether teachers had adequate training and 
had received materials prior to the start of the program. 
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too standardized, and whether 
they supplemented it with other 
materials. About 30 percent of 
teachers agreed that the reading 
curriculum was too difficult (see 
Figure 4-3). Nearly half (48 per- 
cent) agreed or strongly agreed 
that the reading materials were 
too standardized. Teachers felt 
that the curriculum was less 
aligned with the needs of their 
lowest skilled students. Over half 
(57 percent) agreed that they 
“supplemented the curriculum 
with more basic activities and/or 
materials.” A far smaller propor- 
tion, 26 percent, reported supple- 
menting with more advanced 
materials. 

Third-grade teachers were 
somewhat less positive about the 
reading curriculum. Over one- 
third of third-grade teachers felt 
that the reading curriculum was 
too difficult for their students and 
over half agreed or strongly agreed 
that it was too standardized to 
meet individual students' needs 
(see Figures 4-4 and 4-5). We 
might expect third-grade teach- 
ers, who were most likely to re- 
port that the curriculum was 
difficult for their students, to have 
reported supplementing at higher 
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FIGURE 4-2 

Summer Bridge Teachers Were Positive about the Curriculum and 
Materials but Had Concerns about Whether They Were Appropriate 
for Some Students 



The reading materials were useful and 
appropriate for my Summer Bridge students 
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The reading curriculum was well 
organized and of high quality 
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Percent of teachers 

■ Strongly H Agree | Disagree ;|i Strongly 
agree disagree 

Results are from the 1999 Summer Bridge teacher survey. 



The math materials were useful and 
appropriate for my Summer Bridge students 

The math Oirriculum was well 
organized and of high quality 




FIGURE 4-3 

Summer Bridge Teachers Were Less Positive about the Curriculum and 
Materials but Had Concerns about Whether They Were Appropriate 
for Some Students 



1 



The reading curriculum was too difficult 
for my students 

The reading materials were too 
standardized (didn't account for 
individual students' needs or skill gaps) 

1 supplemented the curriculum with 
more basic activities and/or materials 



The math curriculum was too difficult 
for my students 

The math materials were too standardized 
(i.e„ didn’t account for individual 
students' needs or skill gaps) 

I supplemented the curriculum with 
more advanced activities and/or materials 
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■ Strongly H Agree ■ Disagree || Strongly 
agree disagree 

Results are from the 1999 Summer Bridge teacher survey. 
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FIGURE 4-4 

Summer Bridge Teachers Did Not Think the 
Reading and Mathematics Curricula Were Too Difficult 



The curriculum was too difficult for my students . . . 
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Results are from the 1999 Summer Bridge teacher survey. 



FIGURE 4-5 

Summer Bridge Teachers Differed in Terms of Whether 
They Thought the Curricula Were Too Standardized 



The reading and mathematics curricula were too standardized 
Reading 




Mathematics 

Third grade 

Sixth grade 
Eighth grade 
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H Strongly agree flj Agree H Disagree H Strongly disagree 
Results are from the 1999 Summer Bridge teacher survey. 
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rates. However, fewer than half 
(46 percent) of third-grade 
teachers reported supplement- 
ing the curriculum with more 
basic materials, compared with 
approximately two-thirds of 
sixth- and eighth-grade teachers 
(see Figure 4-6). 



Assessmeiil of Pacing 

Teachers are expected to follow 
the prescribed pacing of the daily 
lesson plans and monitors visit 
classrooms to ensure that teach- 
ers cover the material and main- 
tain the pace. This process should 
help ensure that students are ex- 
posed to content and practice a 
range of skills. At the same time, 
teachers may feel frustrated if the 
pace they are expected to main- 
tain does not fulfill what they 
believe their students need. 

Teachers' assessments of the 
curriculum s pacing were mixed. 
Only about half (55 percent) of 
teachers in all three grades were 
satisfied with the pacing of the 
lesson plans (see Figure 4-7). 
Similarly, 56 percent of all teach- 
ers agreed that they could not 
cover all of the materials and top- 
ics required (see Figure 4-8). 
Thus, for about halt of teachers, 
the pacing of the Summer Bridge 
curriculum was too fast. 

In general, survey results tell us 
that Summer Bridge teachers 
were very positive about the over- 
all content of the curriculum and 
the quality of the instructional 
materials. Teachers were less posi- 
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tive about their ability to meet di- 
verse student needs with a one- 
size-fits-all curriculum, and many 
reported supplementing with 
more basic materials. Finally, 
about half of teachers disliked the 
pace of the lessons and reported 
that they could not cover all of 
the topics required. As pacing is 
the aspect of the Summer Bridge 
curriculum where teachers had 
the least flexibility, it is not sur- 
prising that many disliked having 
little control in this area. 

Meeting Individual Students' 
Needs: The Role of 
Class Size and 
Teacher Strategies 

Perhaps because many teachers 
felt that the Summer Bridge cur- 
riculum was too standardized, 
over three-quarters reported that 
they attempted to “tailor instruc- 
tion to the individual strengths 
and weaknesses” of their Summer 
Bridge students (see Figure 4-9). 
Interviews and surveys suggest 
that teachers used three primary 
strategies to accomplish this: capi- 
talizing on small class sizes to pro- 
vide more personal attention to 
students, allotting more class 
time for reading than math- 
ematics, and providing extra 
work outside of class for stu- 
dents who need more practice. 

In interviews, most teachers 
said they appreciated that their 
Summer Bridge classes were 
substantially smaller than their 



FIGURE 4-6 

Sixth- and Eighth-Grade Teachers Were More Likely to 
Supplement the Curriculum with More Basic Materials 



I supplemented the curriculum with more basic/advanced activities and/or materials 
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Results are from the 1999 Summer Bridge teacher survey. 



Summer Bridge Teachers' Assessments of the 
Pace of the Curriculum Were Mixed 



Overall, how satisfied are you with the pace of the lesson plan guides? 
All three grades 
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Results are from the 1999 Summer Bridge teacher survey. 
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school year classes. Teachers emphasized that the reduced class size was 
critical in allowing them to pay more attention to individual students. 
One sixth'grade teacher exemplified this theme. When asked to character- 
ize what students got from dieir experience in Summer Bridge, she replied: 

More attention, first of all. Children blossom with the attention be- 
cause its like they see that, wow, this teacher really cares. It s not that 
the teacher doesn’t care during the regular school year, she just doesn’t 
have the time. I know for intermediate grades, we don’t have helpers 



FIGURE 4-8 

More Than Half of Teachers Agreed They Could Not 
Cover All of the Materials and Topics in Summer Bridge 



I could not cover all the materials and topics required 
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Results are from the 1999 Summer Bridge teacher survey 



FIGURE 4-9 

Over Three-Quarters of Summer Bridge Teachers 
Reported Tailoring Their Instruction 



I tailored my instruction to the individual strengths 
and weaknesses of my Summer Bridge students 




Percent of teachers 

H strongly agree ■ Agree B Disagree I Strongly disagree 
Results are from the 1 999 Summer Bridge teacher survey. 



or assistants. And we don’t 
have that kind of time and 
those kinds of resources 
[during the school year]. 
And they really love that 
extra time, and the teach- 
ers, I mean its a whole atti- 
tude. They’re like, “Wow! I 
can sit down and I can talk 
to student A or student B 
without saying, “Oh, I’ll get 
back to you later,” or 
“Write it down and I’ll ad- 
dress it later.” If they have 
a question now, I say, “Ok, 
hold on. Here I come,” or 
“Come up to my desk.” We 
can address it right then 
and there so the child gets 
the attention and the in- 
struction that they need. 
That’s the best thing about 
the summer program. Class 
size is smaller, is more 
concentrated, and it’s a no 
nonsense approach. No be- 
havior problems; none of 
that is tolerated in summer 
school. You get their atten- 
tion, you get the instruction 
without any interruption. 



This teacher mirrored students’ 
perceptions of increased person- 
alism in Summer Bridge by ex- 
plaining that a smaller class 
allowed her to create a classroom 
environment where students saw 
that she cared and where she 
could provide extra attention and 
focused instruction. In inter- 
views, teachers also mentioned 
that smaller class sizes allowed 
them to group students for in- 
struction more easily, reduce be- 
havior problems, improve 
classroom management, and bet- 




Vfl BEST COPYAVAIUBLE 







RESULTS FROM SUMMER BRIDGE 



69 



ter monitor students’ academic 
performance.’ The small class 
sizes in Summer Bridge appeared 
to counterbalance what teachers 
felt was the curriculum’s weak- 
ness: lack of flexibility to meet 
student needs. 

A second way that teachers tai- 
lored instruction in Summer 
Bridge was by spending more 
time on reading than mathemat- 
ics, either with the entire class or 
with specific groups of students. 
Many teachers we interviewed 
noted that the majority of their 
students needed to meet the test- 
score cutoff in reading but not in 
mathematics. They were also told 
in their training that they could 
spend more time on reading. 
Teachers shifted the focus to read- 
ing in several ways. Some simply 
spent more time on reading 
throughout the summer, while 
others spent more time on read- 
ing only when they felt that their 
students needed it. As one teacher 
explained, “If the majority of the 
children need reading more than 
math, we’ve been told to spend 
more time on the reading. So I 
do spend maybe 10-15, 20 min- 
utes extra with the reading. I just 
go with the flow where I feel they 
need more help.” 

Some teachers also used group- 
ing to adjust time spent on read- 
ing and mathematics for students’ 
individual needs. For example, 
several teachers reported that dur- 
ing the mathematics lesson, they 
asked an aide to do extra reading 
with students who had already 
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met the cutoff in mathematics. 

The third way that many teachers attempted to meet individual stu- 
dents’ needs was to provide extra work for, and attention to, students out- 
side of class time. Given the high-stakes nature of Summer Bridge, students 
certainly had an incentive to do extra work if teachers provided it. On the 
1999 Summer Bridge survey, we asked teachers to report the extent to 
which they provided extra time or work for individual students outside of 
class. Over two- thirds of teachers reported that they assigned extra work 
for individual students (in addition to regular homework) at least once a 
week (see Figure 4-10). About a third reported giving individual students 
extra work almost every day. About half also reported that they worked 
with students before or after class on a regular basis (about once a week or 
more). Approximately 30 percent did so on a daily basis. 

Teachers were less likely to use tutors to work with students outside of 
class time than they were to assign extra work or work with students them- 
selves. Only about a quarter reported having tutors work with students 
outside of class once a week or more. 

Teachers in Summer Bridge often sought out other teachers for infor- 
mation regarding their students. Indeed, 40 percent of Summer Bridge 
teachers reported that they talked to other teachers about their stu- 
dents on a daily basis. Also, about half of teachers in Summer Bridge 
reported talking to students’ parents regularly. 



FIGURE 4-10 



Summer Bridge Teachers' Reports of How Often 
They Extended Effort beyond the Classroom 
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Results are from the 1999 Summer Bridge teacher survey. 
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Eighth-grade teachers were the most likely to report providing extra work 
or help to students. Forty- four percent of eighth-grade teachers, compared 
with about one-third of third- and sixth-grade teachers, reported assigning 
extra work for students outside of class every day (see Figure 4-1 1). Eighth- 
grade teachers were slightly more likely to report working with students 
before or after class (see Figure 4-12). Third-grade teachers reported pro- 
viding the least extra work for students. Given that third-grade teachers 
were also slightly less likely to supplement the curriculum, it is not 
surprising that they were less likely to report that they provided tai- 
lored instruction. 



FIGURE 4-11 

Eighth-Grade Teachers Were the Most Likely to 
Assign Extra Work to Their Students 



This summer how often did you assign extra reading or provide extra work for 
individual students outside of school? 
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Results are from the 1999 Summer Bridge teacher survey. 



32 



33 



44 



— I — 
80 



100 



Almost every day 



FIGURE 4-12 

Eighth-Grade Teachers Were More Likely to Report 
Spending Time out of Class Working with Students 

This summer how often did you work with students individually before or after class? 
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Results are from the 1999 Summer Bridge teacher survey. 
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The Role of Supports in 
Shaping Teacher Practice and 
Perceptions 

What school and classroom char- 
acteristics shape teacher practice 
in Summer Bridge? As we saw in 
Chapter 1, some teachers knew 
many of their Summer Bridge stu- 
dents before the summer, while 
others knew very few. In inter- 
views, some teachers mentioned 
that they felt more efficacious 
when working with students 
whom they knew during the school 
year. For example, one teacher ex- 
plained that she felt hampered 
when helping the two students in 
her class who she l<new the least. 

Interviewer; You know, 
I’ve noticed in class the 
little boy youVe men- 
tioned, he does seem to 
have trouble reading. How 
do you deal with that as a 
teacher, when you ve got 
different levels of ability in 
your grade? 

Teacher; And especially the 
two children I don’t know 
that well. Whereas the other 
children who Ive worked 
with for a couple years, I 
know their strengths and 
weaknesses. But now, with 
this litde fella who is here 
just for the summer, we 
read — his oral reading is 
good — but the comprehen- 
sion is not good at all. If I 
stand next to him and I 
have him read or even do 
the math, he does it very 
well. But on his own, in- 
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dependently, he doesn’t 
perform as well. I don’t 
know him as well; I don’t 
know what the emotional 
problems are, and I just 
try to give him as much 
help as I can without 
spending the whole time 
with him. 

How student and school per- 
formance influenced teacher 
practice in Summer Bridge is an 
open question. On the one hand, 
we might expect teachers with the 
lowest skilled students or in the 
lowest performing schools to be 
the least positive about the Sum- 
mer Bridge curriculum because of 
its grade-level content and pre- 
scribed pace. On the other hand, 
teachers and students in low-per- 
forming schools may benefit from 
having a rich set of instructional 
materials and a more focused en- 
vironment. If this is the case, we 
would expect Summer Bridge 
teachers in low-performing 
schools to be positive about the 
curriculum and learning environ- 
ment of Summer Bridge. 

Finally, we expect that teacher 
behavior may differ across grades. 
Eighth -grade students face the 
most severe consequence if they 
do not meet the cutoffs in Sum- 
mer Bridge — they do not move 
on to high school. It is likely that 
eighth-grade teachers would have 
the most motivated students and 
might be willing to exert extra ef- 
fort on their behalf (see Figure 4- 
13). A curriculum focused on 



skills geared toward ITBS content might be a good fit for these teachers 
because it concentrates on what their students need to learn to graduate. 
Third-grade teachers, on the other hand, may face a more difficult task 
because they are trying to teach a grade-level curriculum to students 
who are less likely to have mastered the basic skills necessary to handle 
the reading comprehension skills central to the ITBS. 

We conducted a multivariate analysis to examine the classroom charac- 
teristics that shaped teachers’ assessments of the curriculum, their efforts to 
meet students’ needs, and their overall evaluation of the learning environ- 
ment in Summer Bridge (see the FILM model described in Appendix H). 
For this analysis, we created three summary measures based on the survey 
items presented in this chapter. The first measure, “teachers’ evaluations of 
the Summer Bridge curriculum,” combines teachers’ responses to ques- 
tions about the Summer Bridge curriculum and its match with students’ 
needs. The second measure, “individualized effort beyond the classroom,” 
combines information on the extent to which teachers sought to supple- 
ment and tailor the curriculum and the extent to which teachers provided 
extracurricular support for students. The third measure, “assessment of Sum- 
mer Bridge learning environment,” draws on teachers’ assessments of stu- 
dent motivation in Summer Bridge as well as their assessments of the learning 
environment in the program. 

For each measure, we examined how teacher reports varied by school 
characteristics (racial composition and school year achievement), by char- 
acteristics of Summer Bridge classrooms (class size, the percentage of 
students the teacher knew before summer, whether the classroom had a 

FIGURE 4-13 

About Half of Summer Bridge Teachers Reported 
That Students Were More Motivated in Summer 
Bridge Than during the School Year 

Students are more motivated during Summer Bridge than during the regular year 
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Results are from the 1999 Summer Bridge teacher survey. 
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Summer Bridge Teacher Measures 

Three measures, "teachers' evaluation of the Summer Bridge curriculum," "individualized effort beyond the class- 
room," and "assessment of Summer Bridge learning environment," were created using items from the 1 999 Sum- 
mer Bridge teacher survey. 

Assessmeot of Summer Bridge Learniri| Environment 

• Students are more motivated during Summer Bridge than during the regular year. 

• I feel that I have a better chance to improve student learning during the summer. 

• The atmosphere in Summer Bridge is more conducive to learning than during the regular year. 

• Students learn at a quicker pace in Summer Bridge than during the school year. 

Evaluation of i!ie Summer Bridge Cumculum 

• Overall, how satisfied are you with the content of the lesson plan guides? 

• Overall, how satisfied are you with the quality of the instructional materials? 

• Does the variety of instructional materials help to meet diverse student needs? 

• The Summer Bridge curriculum is too rigid for my regular teaching style. 

• The reading curriculum was well organized and of high quality. 

• The math curriculum was well organized and of high quality. 

• The reading curriculum was too standardized (i.e., did not account for individual students' needs or 
skill gaps). 

• The mathematics curriculum was too standardized (i.e., did not account for individual students' needs or 
skill gaps). 

individuafeed Effort hmni the Oassroom 

This summer, how often did you: 

• Work with students individually before or after class? 

• Talk to a student's parents about his/her work? 

• Assign extra reading or provide extra work for individual students outside of school? 

• Assign a tutor/aide to work with a student outside of class time? 

• Talk to other teachers about students' performance? 

• I got to know my Summer Bridge students as well as my students during the regular school year. 

• I tailored my instruction to the individual strengths and weaknesses of my Summer 
Bridge students. 

• I supplemented the curriculum with more basic activities and/or materials. 

• I supplemented the curriculum with more advanced activities and /or materials. 
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tutor, and the achievement level of the class), and 
by grade.^ We also took into account teachers’ years 
ot experience and educational attainment, whether 
they had taught Summer Bridge before, and the sub- 
ject they taught during the school year. 

Knowing a large proportion of students, having a 
tutor, and small class size are all resources that would 
provide teachers the opportunity to build relationships, 
with their students and provide individualized atten- 
tion. These first two resources were strongly and posi- 
tively associated with teacher assessments of the 
curriculum in Summer Bridge (see Figure 4-1 4). 
Teachers who knew more of their students, had a 
tutor, and had small class size were more likely to 
report adapting instruction to meet their students’ 
needs (see Figure 4-15). These teachers were also 
more positive about the overall learning environ- 
ment in Summer Bridge as compared to the school 
year (see Figure 4-16). 

Teachers who knew a large proportion of their stu- 
dents may have been more positive about the learning 
environment in Summer Bridge because it was easier 
to adapt the curriculum to meet students’ needs. At 
the same time, knowing their students may have given 
them a better reference point for recognizing height- 
ened motivation during Summer Bridge compared to 
the school year. For example, over half (56 percent) of 
teachers who knew a high proportion of their students 
agreed with the statement that “students were more 
motivated in Summer Bridge than during the school 
year,” compared to only 41 percent of teachers who 
knew very few of their Summer Bridge students. 

The adult-to-student ratio also seemed to shape 
teachers’ experiences and practice in Summer Bridge. 
Teachers who had tutors were more positive about 
both the curriculum and the learning environment 
in the program, and were more likely to provide 
individualized support for students in and outside of 



FIGURE 4-14 

In Classrooms Where Teachers Had a Tutor, 

They Were Much More Positive about the 
Summer Bridge Curriculum 

Effects of Program Characteristics on Teachers* 
Evaluations of the Summer Bridge Curriculum 
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Average PLUS knowing PLUS having PLUS having 
classroom 90% of students a tutor 1 2 or fewer 

students 
(no effect) 



FIGURE 4-15 

Summer Bridge Teachers Who Knew More of 
Their Students and Had a Tutor Were More Likely 
to Provide Extra Help to Students 

Effects of Program Characteristics on 
Individualized Efforts beyond the Classroom 
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Average PLUS knowing PLUS having PLUS having 

classroom 90% of students a tutor 12 or fewer 
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In the average Summer Bridge classroom in 1999, the teacher knew 44% of 
students before Summer Bridge, no tutors were available, and 13 to 17 
students attended. Data are from 1999 Summer Bridge teacher survey. 
Results are based on HLM analysis described in Appendix H. 
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FIGURE 4-16 

Teachers Were More Positive about the Learning 
Environment in Summer Bridge When They 
Knew Their Students and Had a Tutor 

Effects of Program C/raracfer/sf/cs on Assessment 
of Summer Bridge Learning Environment 



7-1 




Average PLUS knowing PLUS having PLUS having 
classroom 90% of students a tutor 12 or fewer 

students 



In the average Summer Bridge classroom in 1999, the teacher knew 44% of 
students before Summer Bridge, no tutors were available, and 13 to 17 
students attended. Data are from 1999 Summer Bridge teacher survey. 
Results are based on HLM analysis described in Appendix H. 



class time. Even with Summer Bridge s relatively small 
class sizes, variation in class size mattered, though the 
effects are small. Teachers with very small classes re- 
ported providing more individualized instruction and 
noted a more positive learning environment. 

Differences across Schools and Classrooms in 
Teacher Practice and Assessments 

There was not a strong relationship between the learn- 
ing gap (the amount that students needed to increase 
their test scores) that students faced in Summer Bridge 
and teacher assessments of the curriculum, the learn- 
ing environment, and their work with students. Teach- 
ers in schools where the average student was far below 
the cutoffs were just as positive about the Summer 
Bridge curriculum and learning environment as teach- 
ers in schools where the average student was closer to 
the cutoffs. 

School-level achievement, however, was strongly 
associated with teacher reports on the three measures. 



Teachers in schools with higher achievement were 
much more likely to report providing individualized 
support to students and to feel that the atmosphere of 
Summer Bridge was conducive to student learning (see 
Figure 4-17).. This was true even after we accounted 
for differences in class size, teacher characteristics, 
achievement, and racial composition. In Chapter 2, 
we saw that students in high-achieving schools also 
had larger learning gains in Summer Bridge. Thus, it 
appears that not only did Summer Bridge students 
learn more in schools with high school year achieve- 
ment, but teachers in these schools also reported more 
effective environments and greater attention to indi- 
vidual students. 

Teacher reports on each of the three measures also 
differed significantly across grades. Eighth-grade teach- 
ers,, in general, were more positive about the Summer 
Bridge curriculum and reported a greater focus on pro- 
viding attention to individual students. The largest 
across- grade differences occurred in teacher assessments 
of the overall learning environment in Summer Bridge. 
Again, we might expect eighth-grade students to be 
more motivated and to change their behavior the most 
in Summer Bridge because they face the greatest cost 
in failing to meet the cutoffs. Motivation is also more 
likely to play an important role in shaping the behav- 
ior of older students than that of their younger peers. 

Some of these across-grade differences, however, 
may also be shaped by the fact that eighth-grade teach- 
ers were more likely to know a large proportion of 
their students. We examined across-grade differences 
in teacher responses both with and without account- 
ing for differences across grades in the proportion of 
students that Summer Bridge teachers knew. About 
one-third of the difference between third- and eighth- 
grade teacher assessments of the curriculum and learn- 
ing environment in Summer Bridge is explained by 
the fact that eighth-grade teachers knew more of their 
students before the start of the program. 
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FIGURE 4-17 



Teachers in Schools with Higher Achievement Were More 
Positive about the Summer Bridge Learning Environment and 
Reported Providing More Individualized Instruction 




Learning 

environment 



Assessment 
of curriculum 



individualization 



I Low-achieving schools 
(less than 25% of 
students scoring at or 
above national norms 
during the school year) 

Data are from 1999 Summer Bridge teacher survey. Results are based on HLM analysis described 
in Appendix H. Resuits shown are after controlling for student and school characteristics. 



High-achieving schools 
(over 40% of students 
scoring at or above 
national norms during the 
school year) 



Moderate-achieving schools 
(25-40% of students scoring 
at or above national norms 
during the school year) 



Did Teachers' Connections to Students and 
Level of Individualization Affect Learning? 

Thus far, our analysis suggests that an important dif- 
ference between teachers in Summer Bridge and teach- 
ers in the regular school year was the extent to which 
Summer Bridge teachers tailored the curriculum to stu- 
dents’ needs and went beyond classroom time to give 
extra assignments and work with students. Because 
teachers who were acquainted with their students were 
more likely to employ these teaching strategies, one 
policy implication of this analysis is the importance of 
recruiting teachers who have prior knowledge of their 
summer students. Other policy implications are de- 
rived from the benefits of providing supports for teach- 
ers such as tutors and small class sizes. A final question 
remains: Do these relationships, behaviors, and pro- 
grammatic components affect learning gains in Sum- 
mer Bridge? 



Using a multivariate analysis, we 
examined how learning gains in each 
grade varied by teacher reports on the 
proportion of their students they 
knew before the program and by their 
emphasis on individualizing instruc- 
tion. This analysis controlled for 
Summer Bridge class size, differences 
across schools in racial composition 
and achievement, teacher character- 
istics, and student characteristics. 
Unfortunately, we cannot link indi- 
vidual teachers to their students for 
this analysis. Our analysis correlates 
adjusted Summer Bridge test-score 
gains for each grade with the average 
teacher reports within schools and 
grades. For example, in the eighth 
grade at School X, we examine the 
correlation between the average ad- 
justed learning gain for eighth grad- 
ers with the average of teacher reports in the eighth 
grade. As we saw in Chapter I, many schools ran mul- 
tiple Summer Bridge classrooms. Thus, our estimates 
of the association between teacher reports and learn- 
ing gains should be considered the lower bound of the 
estimate since we are measuring teacher and student 
links with some error. 

In all three grades, adjusted learning gains in read- 
ing were significantly greater in schools where teach- 
ers reported knowing a larger proportion of their 
students before Summer Bridge (see Figures 4- 18a, 4- 
18b, and 4- 18c). For example, we estimate that in the 
eighth grade, the average adjusted Summer Bridge 
reading gains in a school where teachers knew 90 per- 
cent of their students would be 1 .6 months higher than 
the reading gains in an eighth grade where the teacher 
knew an average number (44 percent) of her students. 
This large effect occurs even after we control for the de- 
mographic and achievement charaaeristics of students 
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FIGURE 4-1 8A 

Students Had Larger Test-Score Gains When 
Teachers Provided More Individualized 
Instruction and Knew Them before Summer Bridge 

Third-Grade Adjusted Summer Bridge Reading Gains 



,30 -1 




Average PLUS knowing PLUS having high 

classroom 90% of students individualization 



Both characteristics are significant at the 90% level. 



FIGURE 4-18B 

Students Had Larger Test-Score Gains When 
Teachers Provided More Individualized 
Instruction and Knew Them before Summer Bridge 

Sixth-Grade Adjusted Summer Bridge Reading Gains 

.40 n 




Average PLUS knowing PLUS having high 

classroom 90% of students individualization 



"Knowing students" is significant at the 95% level. Individualization is not 
significant here. 



and schools, teachers’ qualifications, class size, and 
teacher reports of individualization. The effect appears 
larger in eighth grade than in third. 

Higher scores on individualized effort beyond the 
classroom were also associated with larger test-score 
gains in Summer Bridge. Thus, there is evidence that 
the Summer Bridge program was enhanced when teach- 
ers worked to adapt and supplement the curriculum to 
meet their students’ needs, and when they extended 
their efforts beyond class time. 

There was no independent effect of class size on 
adjusted learning gains, perhaps reflecting the fact 
that there was little variation in class size in Sum- 
mer Bridge, nor was there an association between 
tutors and adjusted learning gains. 

Critical Findings and Questions 

Teachers were positive about the curriculum, ma- 
terials, and small class size. In general, both inter- 
views and surveys indicate that teachers were positive 
about the content of the Summer Bridge curriculum 
and materials. Teachers were also positive about the 
small class sizes. On the 1 999 surveys, we asked teach- 



ers why they thought students who had made progress 
in Summer Bridge did so. Summer Bridge teachers 
most often attributed student progress to additional 
exposure to content (58 percent), small class size (55 
percent), and taking the test more seriously (43 per- 
cent). These choices suggest that teachers were posi- 
tive about three of the primary components of the 
program — its curricular foci, small class size, and its 
high-stalces approach. 

Survey responses suggest that teachers were less in- 
clined to say that the curriculum and structure of the 
program allowed them to do enough to meet diverse 
student needs. Interestingly, relatively few teachers re- 
ported feeling that the curriculum was too difficult. 
Rather, teachers’ responses suggest that although they 
generally thought the curriculum was appropriate, a 
sizeable number stated its “one size fits all” approach 
and fast pacing did not allow them to meet the needs 
of struggling students. One critical question to con- 
sider is whether these teacher responses reflect a gen- 
eral reluctance to employ a mandatory curriculum or 
whether they reflect the particular content and struc- 
ture of the Summer Bridge curriculum. 
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FIGURE 4-18C 

Students Had Larger Test*Score Gains When Teachers 
Provided More Individualized Instruction and 
Knew Them before Summer Bridge 

Eighth-Grade Adjusted Summer Bridge Reading Gains 

.9-1 







Average PLUS knowing PLUS having high 

classroom 90% of students individualization 



"Knowing students" is 99% significant. Individualization is 90% significant. 

For Figures 14A, B. and C: In the average Summer Bridge classroom, the 
teacher knew 44% of students before Summer Bridge, no tutors were 
available, and 13 to 17 students attended. Results are from 1999 Summer 
Bridge teacher survey and student test scores. 

Differences in teachers’ adaptation of the cur- 
riculum and assistance to students outside of class 
were associated with learning gains. There was sig- 
nificant variation in both the extent to which teachers 
adapted and supplemented the Summer Bridge cur- 
riculum and how they provided attention to individual 
students. Almost 80 percent of teachers reported that 
they tailored instruction to the individual strengths 
and weaknesses of their students, and over half supple- 
mented the curriculum with more basic materials. Over 
two-diirds of teachers also reported regularly assign- 
ing extra work to students outside of class, while about 
half reported that they worked with students outside 
of class on a regular basis (once a week or more). What 
is important, however, is the extent to which these extra 
efforts are associated with adjusted learning gains in 
the program. Summer Bridge students who attended 
schools where teachers reported exerting extra effort 
had larger learning gains. An important policy ques- 
tion is whether such activities could be further encour- 
aged, both in the Summer Bridge program and in 



summer programs in general. Summer Bridge teach- 
ers seem to have been highly sensitive to the messages 
they received in training about adapting the curricu- 
lum. For example, in interviews, teachers reported 
spending more time on reading than mathematics, a 
strategy that had been recommended in training ses- 
sions. Summer programs can send the message to 
teachers that meeting individual student needs is as 
important as covering the required content and could 
provide teachers extra supports and training in this 
area. We also find that extra supports, specifically 
the addition of tutors to classrooms, enhanced teach- 
ers’ abilities to provide more individualized atten- 
tion to students. 

The most important finding in this chapter, how- 
ever, is that there are strong associations between 
teacher efforts, staffing arrangements, and learning 
gains. Teachers who knew a large proportion of their 
students before Summer Bridge were more likely to 
report adjusting instruction to meet student needs and 
were more likely to work with students outside of class. 
In turn, high individualization was associated with 
larger test-score gains in the program. Teachers who 
knew more of their students may have been more likely 
to individualize and adjtLSt the curriculum because they 
already had a relationship with students and because 
they may have felt more effective in identifying and 
meeting students’ needs. In fact, even after we account 
for teacher reports of individualization and extra sup- 
port, their prior knowledge of their students has a 
strong and direct association with Summer Bridge test- 
score gains. 

Teacher assessments of Summer Bridge differed 
by school achievement, grade level, and by their 
prior knowledge of students. Finally, we find that 
teacher assessments of Summer Bridge mirrored pat- 
terns in our earlier analysis of learning gains. In Chap- 
ter 2, we found that third graders had smaller test-score 
gains in Summer Bridge than eighth graders. We also 
found that students in schools with higher school year 
achievement had larger test-score gains in the program. 
Similar differences were found in teachers’ assessments 
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of Summer Bridge. Eighth-grade teachers were more 
positive about the curriculum and reported greater at- 
tention to those behaviors — individualizing instruc- 
tion and providing extra support outside of class — that 
were associated with greater test-score increases. 
Eighth-grade teachers were also the most positive about 
the Summer Bridge learning environment and were 
more likely to have known their students before the 
program. Third-grade teachers, on the other hand, were 



the most negative abdut^the curriculum and were less 
likely to have known their students before the sum- 
mer. Third-grade teachers were also less likely to 
supplement the curriculum or provide extra sup- 
port to students. Thus, third-grade Summer Bridge 
teachers were the least likely to have characteristics 
and report practices that were associated with stu- 
dent learning gains. 
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CHAPTER 




A Closer Look at Instruction 



n the previous chapter, we found that many teachers reported tailoring 
their instruction to meet students needs. How much variation was there 
in Summer Bridge instruction? There are at least three ways that teach- 
ers might vary their instruction. First, they could differ in how much they 
used the curriculum and followed the daily lesson plans. Second, they could 
differ in how they enacted the curriculum. Although the Summer Bridge 
curriculum gave teachers a basic outline and suggested approaches for in- 
troducing topics, they did have flexibility in how to structure specific les- 
sons, what skills to emphasize, and how to teach those skills. Third, teachers 
could differ in how they grouped students for instruction — the extent to 
which they had students work in small groups, individually, or as a whole 
class. In this chapter, we take a closer look at instruction in Summer Bridge, 
drawing on detailed classroom observations conducted in 12 schools in 
2000. Was there variation in how teachers followed and used the Summer 
Bridge curriculum? Did the mandatory curriculum lead to more com- 
monalities than differences in instruction? Finally, were differences in 
instructional environ ments across classrooms associated with differences 
in learning gains? 

Teachers Followed the Daily Lesson Plans and Stayed on Topic 

The 2000 Summer Bridge reading curriculum had 29 lessons for third and 
sixth graders and 34 for eighth graders. Classroom observations in the 12 
schools began in the second week of the program and ran through the fifth 
week. Within those four weeks, researchers conducted approximately 140 
observations in reading and mathematics (about 280 total), including 12 
different daily lessons in each grade and subject. Thus, over the course of 
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Data Used in This Chapter 

This chapter draws on results from a field study 
of classrooms in Summer Bridge during the year 
2000. The field study was an intensive qualita- 
tive examination of instruction in Summer 
Bridge. Twelve schools were included, with each 
school assigned a field researcher. Researchers 
observed Summer Bridge reading and math- 
ematics classes four times in one third-, sixth-, 
and eighth-grade classroom in each school for 
a total of approximately 1 40 observations in each 
subject. During class time, observers maintained 
a log of what occurred in the class, including 
detailed information about the content of the les- 
sons, teachers' instructional style, how much 
time was spent on different activities, classroom 
configuration, classroom management, and 
student behavior. In-depth teacher interviews 
were conducted with 37 of the 40 participat- 
ing teachers at the end of the summer. Inter- 
views included questions about teachers’ 
opinions of the level of their students' aca- 
demic risk, class size, the curriculum and ma- 
terials, and instructional strategies. 

We selected pairs of one high-achieving and one 
low-achieving school that were from the same 
area and had similar demographics. In selecting 
the schools, we used empirical Bayes estimates 
of learning gains in mathematics and reading 
from a hierarchical linear model that controlled 
for school-level characteristics such as racial 
composition, achievement, percent of students 
excluded from testing, and percent of students 
meeting the promotional cutoff scores. Analysis 
also controlled for student characteristics, in- 
cluding socio-economic status, gender, race, 
age, and prior achievement. 







the summer, we were able to track how many of the 
40 teachers we observed kept pace with the reading 
curriculum by teaching the lesson intended for that day. ' 

Throughout the summer, over 90 percent of ob- 
served teachers remained on pace or taught the daily 
lessons at a pace faster than what was prescribed. In 
the second week of Summer Bridge, 95 percent were 
teaching the lesson plan assigned for that particular 
day or were within a day of that lesson. One teacher 
was over a day behind the expected pace, and one was 
over a day ahead. Despite the fact that in Chapter 4 
we found that many teachers felt the pacing of Sum- 
mer Bridge was too fast, by week five of the program, 
1 8 percent of teachers were actually teaching a day or 
more ahead of the prescribed lesson, and 70 per- 
cent were within a day of the expected lesson (see 
Figure 5-1). In only four lessons over the summer 
were teachers “off curriculum,” teaching content not 
prescribed in a daily lesson. In this very small propor- 
tion of lessons, teachers modified the content for a 
day or two because they felt their students had not 
quite grasped a concept or because they felt that the 
curriculum did not include adequate instruction on 



FIGURE 5-1 

Most Teachers Stayed on Schedule with 




Week 2 Week 3 Week4 Weeks 



Week of Summer Bridge 

Based on classroom observations conducted in 2000 

■ Off curriculum B Within a day 

i Over 1 day behind Over 1 day ahead 

Week 2 observations included 37 teachers; week 3, 34 teachers: week 4, 
35 teachers; and week 5, 33 teachers, 
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an important topic. Thus, we find extensive evidence 
that teachers not only used the daily lessons, but they 
were able to maintain the expected pace of instruction. 

Variation in Instruction: 

How Teachers Enacted the Curriculum 

During the course of each observation, field research- 
ers maintained a comprehensive log of what occurred 
in the class, including detailed information about the 
content of the lessons, teachers’ instructional style, how 
much time was spent on different activities, classroom 
configuration, classroom management, and student 
behavior. Each observation was coded to explore varia- 
tion in four aspects of teaching. Namely, the extent to 
which teachers: 

• Kept pace with the daily lessons, implemented 
the instructional foci, followed the guidance 
and suggestions provided in the daily lesson 
plans, and used the instructional materials. 

• Introduced and provided instruction on con- 
cepts in a clear and accurate manner versus 
simply asking students to do the required 
assignments without substantial guidance 
or instruction. 

• Engaged students in the class and encouraged 
their participation by asking questions. 

• Monitored students’ understanding of the 
lessons; provided substantive feedback on stu- 
dent participation and work; and geared in- 
struction to individual needs, the needs of 
the entire class, or both (see Appendix I for 
methodology). 

Once each observation was coded, we grouped lessons 
into four categories that represented common themes. 
The category with the highest quality lessons was char- 
acterized as having “tailored instruction.” At the low 
end, lessons were characterized as having “insufficient 
instruction” (see Figure 5-2). 



FIGURE 5-2 



The Majority of Summer Bridge Instruction 
Was Sufficient 




Tailored Sufficient Minimai Insufficient 

instruction instruction instruction instruction 



H Reading B Mathematics 

Instructional categories for reading and math lessons observed in Summer 
Bridge 2000. In reading, 136 teacher observations v,/ere used in this analy- 
sis, In math. 139 teachers were observed. 

Groups 1 and 2: Tailored and Sufficient Instruction 

In close to 20 percent of Summer Bridge lessons, the 
teacher created a classroom environment that we refer 
to as tailored instruction. In these classrooms, teach- 
ers followed the daily lesson, provided clear and accu- 
rate instruction about the topic, and had meaningful 
and constructive interactions with their students. These 
teachers pushed students to think and made sure that 
they understood the lesson and were learning. They 
often went beyond the instructions in the daily lesson 
to present topics in multiple ways and structured the 
lesson so that strong students were moving forward 
while struggling students received extra attention and 
support. In Lesson Example 1, Ms. J provides an ex- 
ample of tailored instruction. 

Ms. Cs lesson (also in Lesson Example 1) provides 
an example of the majority of lessons we observed in 
Summer Bridge. We refer to these lessons as providing 
“sufficient instruction.” Just over half of reading les- 
sons and 40 percent of mathematics lessons fell into 
this category. These teachers structured class time and 
taught the instructional focus. Even though they tried 
to engage students in the work and interact with them, 
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Lesson Example 1: A Day of Challenging and Engaging Students 
(Tailored Instruction) Versus Just Teaching the Lesson (Sufficient Instruction) 

Eighth Grade Reading, Lesson 8: inferring Character Traits and Feeiings 

Read the introduction and selection on pages 66-69 in advance. Select words from the text with which students may not be familiar to 
present as vocabulary words. Write on the chalkboard a sentence using each vocabulary word. Underline the word. Discuss the 
pronunciation and definition of each word. 

Introduce the selection to the students by following the guidelines on page 66 of the Teacher's Edition. Guide students to understand how 
they can infer the meaning of an unfamiliar word. Have students read page 67 silently. Ask questions to check for comprehension. Guide 
students to infer the meanings of the underlined words. Allow students to read page 68 silently pausing between the change of settings 
to check comprehension. Complete reading the selection. Do "Recalling Facts" and Interpreting Facts" as group exercises, discussing 
responses for each section before moving to the next Note areas of difficulty for further skill development Prepare SRAs for students. 
Source: Chicago Public Schools Summer Bridge Teacher Handbook 1999, Reading Grade 8. 



Ms. J's Reading Lesson (52 minutes) 



Breakdown 


16 minutes individual reading, 10 minutes students reading aloud, 26 minutes on SRA and skill cards 


Aides 


1 monitor. 1 aide 


Students 


9 total (4 tardy) 


Code 


Tailored Instruction 


On Board 


Instructional Focus: Infer Character Traits and Feelings 

Vocabulary 

cluttered: John's desk was so cluttered, 1 couldn't find a pen. 

strewn: After the storm, things were strewn everywhere. 

animated: The actors were very animated after the play. 

emporium: Joann bought her wig at the Hair Emporium. 

hilarious: When Pam stepped on the eggs in the lunchroom, it was hilarious. 



Ms. J begins class by reading the instructional focus. The teacher asks for students to define "infer," and one student offers "to figure out." 
Ms. J also asks the class to give examples of "traits" and a student says "smart." Ms. J then introduces the short story, "This is a story 
about a boy who loves to take pictures. It is his hobby." She asks the class to define “hobby." The class then reads the first sentence of 
the story together, in which the word "compensate" appears. Students struggle to understand the concept, and Ms. J provides several 
examples to help them. Finally, the students look up "compensate" In the dictionary, and they discuss several meanings of the word. Ms. J 
then turns students’ attention to the vocabulary on the board. She says, "Here are some other words you might have a hard time with.” 
Eight of the nine students listen attentively as Ms. J walks them through each word and its meaning. (13 minutes) 

Ms. J then directs the class to their books. She says the story is called "Getting into Focus" and provides the meaning of the word "focus." 
She tells the students to read the first page of the short story and write down vocabulary words they cannot determine from the context 
of the story. Each of the students reads silently as aides remind them to write down words they don't understand. (7 minutes) 

The teacher asks if everyone is finished with the story. They are. Ms. J asks students to share vocabulary words that stumped them. 
Several students have written "preoccupied." Ms. J has the students read the sentence before and after the word "preoccupied" to put it 
Into context. After this, several students offer incorrect definitions of "preoccupied." Ms. J gives several other examples of the word until 
two students blurt out the correct definition. Another word that troubled students is "disheveled." Ms. J directs them to the word 
"cluttered" on the board and explains that the author Is using many different but similar words to describe the character's room. Students 
and teacher discuss each vocabulary word. At this point, all nine students are engaged in offering answers to help each other to 
understand the difficult ones. Students suggest both correct and incorrect definitions. Ms. J listens to the responses before providing the 
correct answer. 

Ms. J then asks the class to describe the main character. One student says he is "punctual" because he has several jobs. Ms. J presses 
the student to explain why knowing the character has several Jobs would lead him to infer punctuality. "What is it about him having 
several jobs that makes him punctual? Is he busy? Could he keep several jobs without being on timeT Other students offer comments. 

The teacher explains that the student inferred the character is punctual from other aspects of the story. As a group, the whole class then 
tackles other inference questions in the same manner. (11 minutes) 
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Ms. J instructs the students to read the second half of the story. Teacher and students summarize the basic facts and story line. The story 
is left open-ended, so students and teacher make inferences and discuss the probable ending. Using the same procedure as before, 
students and teacher walk through challenging vocabulary words from the second half of the story. The entire class actively continues to 
participate in this exercise. (2T minutes) 

Ms. C's Reading Lesson (47 minutes) 



Breakdown 


41 minutes students read aloud, 6 minutes on SRA and skill cards 


Aides 


1 monitor 


Students 


17 total (0 tardy) 


Code 


Sufficient Instruction 


On board 


scanned emerging emporium exasperated 




disheveled compensate illusion animated 



Ms. C tells students that today they will be inferring character traits and feelings. She briefly asks the students to define "infer" and how 
we can make inferences from a story using the clues from the context. Students are Instructed to take out their books. One student 
begins to read the short story aloud. The teacher asks the class what "compensate" means. When the students cannot answer, the 
teacher has them reread the sentence containing the word. When students still cannot answer. Ms. C follows up by asking "What if you 
don't turn in your homework and I compensate by giving you more?" Other students read passages of the short story aloud until each has 
had a turn. As the students read aloud, Ms. C asks the class inference questions about the characters in the story. "Is the girl afraid? 

Does she have high self-esteem7 Throughout, the teacher corrects any mistakes and praises good reading. Fourteen of the 17 students 
are engaged and on task at this point. 

When a vocabulary word surfaces in the story, the reading stops as the class discusses it. Ms. C encourages the class to use context 
clues to understand the vocabulary. Sixteen of the 17 students appear engaged. When the class reaches the word "exasperate" in the 
story, Ms. C asks the group to provide synonyms. Ms. C suggests "frustrate" as an example. Ms. C asks which students keep their rooms 
clean and what their parents say about it. The teacher reminds the class, "We make Inferences all the time." (19 minutes) 

Ms. C directs the students' attention to the vocabulary words on the board. The teacher asks the students to read each word aloud and 
then define it. Sixteen of the 17 students call out answers: for "scanned" — "look around" and "a cursory look." The teacher praises 
particularly appropriate responses. The teacher tells students that contextual clues are often in sentences later in the story. Students are 
given homework exercises and are permitted to begin them in class. The lesson ends with the students applauding themselves for their 
good work. (28 minutes) 

Eighth Grade MathematicSr Lesson 8: instructionai Focus - interpreting Muitibar Graphs 

The Instruction and problem solving in these pages will enable students to read and interpret multibar graphs, compare three sets of data, 
make generalizations about graphed data, estimate graphed values. 

Explain that a multibar graph can be used to compare changes in two or more sets of data. Give students the example of a school raising 
money for a trip. Each grade level will sponsor three money-raising events: a penny drive, a bake sale, and a walkathon. A multibar graph 
would clearly show the amount of money raised by each grade for each event You may wish to use Transparency 1 1 as you go over this 
page with students. Ask students the list of suggested questions. 

As students work with graphs, suggest that they pay special attention to the form an answer will take. In problem 3, this would be a 
percent, not a grade or a time or a school. 

Source: Chicago Public Schools Summer Bridge Teacher Handbook 2000, Mathematics Grade 8. 

(continued on next page) 
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Lesson Example 1 [continued] 

Ms. J's Mathematics Lesson (S3 minutes) 



Aides 


1 monitor, 1 aide 


Students 


9 


Code 


Tailored Instruction 



Ms. J begins class by posting a sample multibar graph. She walks the class through the title, how to read intervals, how to read values 
between identified intervals and each of the different components of a graph before reading the questions from the day's math activity. 
Ms. J reads each of the questions, the students call out the answers, and then she demonstrates each answer in more detail for the 
class. Ms. J periodically checks that each of the students is following along. (5 minutes) 

Ms. J tacks a piece of white paper at the front of the room. She then divides the class into groups and passes out graph components to 
each group, (such as the x and y axes. data, pieces of construction paper cut to the length of data, etc.) Students are given directions and 
instructed to put together a graph. After a few moments of confusion, the students complete the assignment. Ms. J tells the class to take 
out their books and work in groups on the problems on page 30. The teacher's aide circulates and answers questions as the students 
work out the problems. Every student is engaged and working busily. As each student finishes the assignment, they bring their work to the 
teacher. She checks it over, briefly discusses wrong answers, and sends each student back to correct mistakes. Ms. J asks a few 
students who understand graphing to assist two students who are struggling. (35 minutes) 

Ms. J brings the class together again, and she directs the class's attention to one problem that everyone missed. The problem involves a Y 
axis labeled "time" and the question asks for the slowest activity. The class had chosen the lowest rather than the highest bar. She goes 
through the problem carefully. It is clear the students understand, A murmer of "Oh. 1 should have gotten that one" runs through the class. 
Ms. J reminds the students to read and understand each question on the Iowa test (ITBS) before they answer. Ms. J then goes over the 
remaining answers with the class, calling on students who aren't raising their hands to answer. When the class disagrees, she has the 
students offer explanations to each other. (23 minutes) 



Ms. C's Mathematics Lesson (61 minutes) 



Aides 


1 monitor 


Students 


13 


Code 


Sufficient Instruction 



Ms. C begins class by asking students to take out their math materials. "We have been talking about graphs." One student groans, calling 
the lesson "boring." Ms. C tells the students she doesn’t determine the curriculum and besides, "This is for your own benefit." Moving on, 
she describes the lesson's objectives: to read and interpret graphs, to compare three sets of data, and to estimate graphed values. She 
draws a vertical and horizontal axis on the board as one student reads aloud an example from the textbook. The class discusses very 
briefly the types of graphs the students prefer. The students continue to read the questions and answers aloud for the next 30 minutes. 
The teacher interrupts the reading twice: once to voice her disagreement with an answer to one of the questions, and a second time to 
lecture the class on the importance of a good night's sleep to maintain energy for class. As the students read aloud, Ms. C goes to the 
board to clarify one of the examples in the book, and then to a student's desk to explain this point individually. Eleven students are paying 
attention and two are not listening. Ms. C solicits student input on question #6, page 30. She asks one student to go to the board to 
estimate an answer. (43 minutes) 

The teacher tells the class to work in groups to complete the remaining problems. She then walks around to provide direction to students 
with questions. Again, 11 students are engaged and two are not. Near the end of the lesson. Ms. C explains one of the more difficult 
problems to the entire class. The math class concludes with Ms. C reminding students that the Iowa Test (ITBS) is in three weeks. (18 
minutes) 
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they did not go beyond the 
instructions in the daily 
lessons to ensure that all 
students understood the 
concepts. While Ms. J 
pushed her class to under- 
stand the concept of infer- 
ence, Ms. C had students 
make inferences without 
helping them to under- 
stand the connection be- 
tween the questions she 
was asking and the larger 
concept of inference. xMs. 

J helped students learn vocabulary words by putting 
them in context. She had students identify the words 
they did not understand and then worked 
collaboratively with the class to define the words and 
teach students skills to continue to build their vocabu- 
laries on their own. Ms. C, on the other hand,, named 
problematic words herself and generally elicited stu- 
dents’ participation without ensuring that they were 
gaining knowledge or building skills. 

In mathematics, Ms. J and Ms. C also differed in 
their enactment of the curriculum. Although both 
taught multi-bar graphs, Ms. J carefully and thoroughly 
explained the topic, used peer tutoring and had stu- 
dents work in small groups on a range of tasks, circu- 
lated to support students in their work, and brought 
the class together to clarify and re teach an aspect of 
the lesson. Ms. C also provided basic instruction 
that was clear and accurate; however, while Ms. J 
used small groups for much of the class, Ms. C had 
students read questions and answers aloud for al- 
most half of the lesson. 



“minimal instruction.” 
These teachers did not ac- 
tively engage students in 
the lesson, nor did they 
provide them with feed- 
back. Ms. A in Lesson Ex- 
ample 2 illustrates this 
type of instruction. Al- 
though the lesson was 
structured and students 
read aloud, defined 
words, and later worked 
independently, the in- 
structional foci were not 
really mentioned and the teacher did not provide 
instructions* or support students as they completed 
the work. 

Group 4: Insufficient Instruction 

In approximately 23 percent of the Summer Bridge 
classes we observed, the lesson was virtually not imple- 
rriented. We refer to this group as “insufficient instruc- 
tion.” In these lessons, little or no time was devoted to 
instruction. There were often extensive periods of down 
time, and teachers provided litde structure and inad- 
equate classroom management. Lesson Example 3 il- 
lustrates this type of instruction. Mr. S s lesson included 
substantial amounts of down time and non-instruc- 
tional interruptions; poor classroom management; and 
unclear, fragmented and inaccurate instruction. Mr. S 
did spend some time talking about division, the in- 
tended instructional focus of the lesson, but he did 
not present the content in a clear or coherent manner 
and students were disengaged for much of the class. 



'Only 20 percent of the time, 
however, did we observe lessons 
where teachers went beyond pro- 
viding clear and accurate in- 
struction to engage students 
actively in the learning process, 
challenge them, and ensure that- 
they were mastering concepts. 



Group 3: Minimal Instruction 

In a little over 10 percent of lessons, teachers created 
an environment where students worked on assignments 
and used materials related to the instructional focus, 
but provided little instruction and little or no support 
or guidance. We refer to these classes as providing 



Observing Instructional Quality 

The majority of lessons (approximately two- thirds) fell 
into our sufficient or tailored instructional categories. 
Only 20 percent of the time, however, did we observe 
lessons where teachers went beyond providing clear 
and accurate instruction to engage students actively 
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Lesson Example 2: Minimal Instruction 

Sixth Grade Reading Lesson 21 
Instructional Focus: 

Interpret non-literal language, deduce meanings not explicitly stated in text, and identify author's point of view. 

Teacher Directed Activity: 

Review previous day's homework assignment. Place instructional focus on chalkboard and discuss briefly with the students. 

Share with students that the author uses Figurative language to add color and humor to "The Will and the Way" and to help readers 
picture in their minds what is being described. 

Place the following terms on the chalkboard: simile, personification, metaphor, and hyperbole. Pronounce each term and discuss by 
turning to p. R6I; share focus information with students to see how the author uses figurative language to add color and humor to the 
selection. Use the visual/spatial model on p. RSI to model examples of simile, metaphor, personification, and hyperbole. 

To determine if students are able to understand the use of figurative language, ask them: What does the author mean when she says that 
Paul "carved his own road”? To check students' understanding, have them define four kinds of figurative language. Have students use 
examples of figurative language. Then have students use two kinds of figurative language in sentences that explain short-range future 
plans, such as what they are going to do first when their vacation begins. 

Source: Chicago Public Schools Summer Bridge Teacher Handbook 1999, Reading Grade 6. 



Ms. A's Reading Lesson (88 minutes) 



Aides 


No Aides 


Students 


12 


Code 


Minimal Instruction 



Classroom locked, students are waiting outside. After five minutes Ms. A arrives. Students pile into the classroom while the teacher 
finishes eating her breakfast. (11 minutes) 

Ms. A reads a passage from the textbook and asks if everyone finished their homework. Teacher sits down at her desk at the front of the 
room and says, "Question one, ’simple' means?" Several students shout out their answers. Ms. A explains that the meaning is not based 
on the definition but on the context of the passage. Teacher asks students to read out their answers to each question. Teacher asks, 
"What do you want to know about the passage?" Students offer several answers. Ms. A continues, "What did you learn from the 
passage?" Students again offer several answers. (9 minutes) 

Ms. A reads a paragraph and then asks students if they have ever been to Los Angeles and if they saw the LAX theme building. Students 
respond. The teacher then asks a student to continue with the next paragraph. Occasionally, Ms, A asks students for definitions of words 
located in the paragraph. After the student is finished, another student reads, and the cycle continues for 40 minutes. (40 minutes) 

Ms. A announces, "It's now SRA time!" She walks over to her desk and takes out a stack of SRA cards and begins distributing them to the 
students. While some students work diligently on SRA exercises for the remainder of the class time, other students attempt to talk among 
themselves. Noticing the disruptive behavior, Ms. A separates the students and tells them to work on their SRAs. (28 minutes) 
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Lesson Example 3: Insufficient Instruction 

Third Grade Mathematics Lesson 10 
Assigned Lesson Plan 

Just the Facts! I Have students turn to p. 50 in the Test Best book. Read and discuss the directions with the class for Try This'’ and Think It 
Through." Assign problems 1 -6 for practice. Check questions and answers with the class. 

Daiiy Challenge 

Susan has 8 dimes and 20 pennies. She divides them equally among three friends and herself How many dimes does each get? How many 
pennies does each get? How much money does each ha ve? 

Turn to p. 12 in On My Own. Pass out 15 counters to each student. Work through problem 1 on the top of the page with the class. Make 
sure the students understand what they are doing. Have the students write a multiplication sentence for each division sentence. Assign 
problems 2- 1 1 for independent work. Check. 

Write the following problem on the chalkboard: 21+7=? 

Ask the students to look up the following words in "Math Tools”: dividend, divisor, quotient, factor, and product. 

Ask for volunteers to read definitions. Ask: What are the names of the digits in the problems on the chalkboard? 

21 = dividend, 1 = divisor, and 3 = quotient Remind the students that multiplication and division are inverse operations. Since 3x7 = 21, 
then 21+7=3. Review the vocabulary on the top ofp. 74 in On My Own. 

Demonstrate on the chalkboard how to use an array to show division. Show an array of 12 and solve 12/ 3. 

Step 1 Step 2 

Start with 12 shown in this array. Separate into 3 equal groups of four 

^m********** *» * * 

* * * 

* # * * 

So, 12 + 3 = 4 

Work through problem 5 with the students. Assign problems 6-16 as independent work. Check. Have the students open Take Another Look 
to p. 76. Work through the example on the top of the page with the students. Ask for a volunteer to read the definition of a factor and 
product from "Math Tools." Ask for volunteers to provide the answers to problems h11 onp. 76. Remind the students that before you can 
choose an operation to solve a problem, you must decide what the problem asks you to find. Carefully review the “Remember" with the 
students. Work through problem 1 with students. Invite volunteers to read and solve problems 2-4 on the chalkboard. 

Source: Chicago Public Schools Summer Bridge Teacher Handbook 1999, Mathematics, Grade 3. 



Mr. S*$ Mathematics Lesson (72 minutes) 



Aides 


No Aides 


Students 


12 


Code 


Insufficient Instruction 



Mr. S tells students to take out a piece of paper to write down homework for the day, which is written on the chalkboard. The teacher 
walks to the back of room to sharpen pencils for the class. He teds one student to sit at the side of the class and look up 100 words in the 
dictionary, and gives him one hour to do it. The boy makes a face at the teacher and Mr. S extends the assignment to 200 words. 



(continued next page) 
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The teacher distributes Skill Poweno students and tells them to raise their hands if they have their times table charts. He makes the eight 
students who do not have their charts stand and tells them that they will get a "zero" (It is unclear what the "zero" applies to). "How do you 
plan on learning today’s work If you don't have your charts? You cannot participate in today's lessons if you don't know your tables." 

Mr. S tells the students to take out their books; he checks one student's homework and explains to her that 2 x 3 = 5, not 6 {sic). The rest of 
the class is off task (e.g., 3 girls chatting, 1 boy looking out the window, etc.) while he talks to the girl. The teacher goes to the next student 
to check her homework. He tells the students who are still standing to write their times tables as he corrects homework. Three of the 
eight start working on their tables while the others continue talking to each other. The teacher continues to grade students' homework 
one by one. Now only one student is working on his tables. Mr. S shouts at one student, The pencil is right in front of your face. Write your 
times tables. What excuse do you have now?” He yells at another student who is not doing her tables, Tou've got something else to say?" 
The teacher assigns her to write 200 definitions from the dictionary. One student starts dancing behind the teacher. At this point, except 
for the 2 students writing definitions, all students are off task. (30 minutes) 

Mr. S writes "21 -s- 7 = ?" on the chalkboard and asks the students to solve it. Students are off task (e.g., 3 boys behind the teacher's back 
are moving around. 1 boy is under the desk, 2 girls are chatting). When students do not respond, the teacher repeats question. Several 
students attempt wild guesses to answer the question; every answer was wrong. 

Mr. S writes a new question on the chalk board: "1 + 4 = ?“ He stops writing and yells at one student who Is talking to a friend. "(Student 
Name], open your mouth again and I'm going to hurt you!" The teacher asks students what a dividend is. Nobody answers initially, then 
one boy reads the answer from his textbook. The teacher tells students to take out their textbook and tell him what a divisor is. Students 
answer, The number after 'by' is the divisor (e.g.. divide 4 by 2)." Mr. S writes the answer on the board and instructs students to look up 
the word "quotient" in their books. The teacher writes on the chalk board; 

21 + 7 = 3 (21 = dividend, 7 = divisor, and 3 = quotient) 

The teacher stops writing and approaches a boy. He grabs the boy by the hand and makes him leave the class. He closes the classroom 
door and tells the class, "You're going to learn or you're going to pay the price in three weeks when your Iowa Test [ITBSj comes, and 
you'll know if you pass or fail You don't pass math and you fail; you fall English, you fall You’ve got to pass both!" 

Mr. S tells the class to write down what he has written on the board and asks the class, "What is a factor?" One girl reads the definition 
from her book. As students copy the information on the chalkboard, the teacher asks them to give an example of factors. Various students 
reply 5 x 5, 3 x 6, 2 x 10, etc. The teacher then says, The answer you get is the product: 3x6-18. Factors are numbers you multiply 
together to get an answer." The teacher tells students to write down the numbers on the chalkboard. 

Someone comes into the classroom with a stack of emergency forms. The teacher explains the form to the class and the importance of it; 
Mr, S tells students how a year earlier a car drove onto the playground and hurt some kids, so they should fill It out In case that happens 
to them. Mr. S tells students to open their books and.asks them how to solve the first question. When no students answer, he tells them to 
look at their times tables. The teacher asks a student for the answer. When the student does not reply, Mr. S says, "Say, 'I don't know'." 
Student complies and the teacher moves on to another student. Most of the class is off task - 8 students did not even have their books 
out. The teacher calls on one of the students writing definitions to answer question 1, She stops writing and takes out her math book. 
While she is looking at the question, another student offers an answer. The teacher replies, "Shut up! If you misbehave one more time, you 
cannot come to summer school anymore!" The teacher goes on to the next questions and answers them for the class. The teacher 
finishes and the students get ready to leave. (42 minutes) 
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in the learning process, challenge them, and ensure 
that they were mastering concepts. We observed higher 
quality instruction in reading than in mathematics. 
Twenty-seven percent of mathematics lessons, com- 
pared with only 18 percent of reading lessons, were 
categorized as insufficient instruction. This difference 
may reflect greater teacher expertise in reading instruc- 
tion or a decision to place more emphasis on reading. 
The higher levels of insufficient instruction coidd also 
be attributed to the scheduling of mathematics les- 
sons. In most cases, mathematics was taught after read- 
ing; classroom management may have been more 

Summer Bridge classrooms were 
characterized by high levels of 
skills practice and instruction fo- , 
cused. on the prescribed topics. In 
general, we found that instruc- 
tional time ■ was geared toward 
building students’ basic skills. 

difficult during this period because teachers and stu- 
dents were losing stamina. It is also possible that the 
reading curriculum was designed in a way that allowed 
teachers to implement lessons more smoothly and co- 
herently; the mathematics curriculum required teach- 
ers to switch topics more frequently, and teachers 
reported that they struggled with the pacing in math- 
ematics more than in reading. 

We also observed differences in instructional qual- 
ity across grades. Reading instruction among eighth- 
grade teachers in our sample was of lower quality than 
what we observed in the third and sixth grades. Less 
than two- thirds of eighth-grade reading teachers pro- 
vided tailored or sufficient instruction (the top two 
categories) compared with over three-fourths of sixth- 
and third-grade teachers. Mathematics instruction in 
sixth grade was lower in quality than in other grades 
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as well. We do not know whether this pattern is gen- 
eralizable to other Sumrner Bridge classrooms or is sim- 
ply an anomaly in this small sample of schools. 

Differences in Instructional Activities 
and Grouping 

Because teachers followed the curriculum closely, most 
instruction in Summer Bridge included a review of 
homework and an introduction to the lesson followed 
by instruction, skills practice, and then some review. 

In examining instructional activities in Summer Bridge, 
we drew, on a coding scheme developed by Consor- 
tium researchers.^ Instructional time includes time 
spent lecturing on the instructional focus (knowledge 
acquisition), having students practice skills (skills prac- 
tice), reviewing work they had done previously (re- 
view), taking tests (testing), and working on in-depth 
problem-solving activities (understanding). Time spent 
preparing for instructional activities (set-up) included 
both instructional and non-instructional time. In our 
Summer Bridge observations, approximately one- third 
of set-up time included instruction.*^ Additional non- 
instructional time includes time where the class is dis- 
rupted and there is no formal activity (down, 
disrupted), and time spent taking attendance and other 
non-academic tasks (housekeeping). 

Summer Bridge classrooms were characterized by 
high levels of skills practice and instruction focused 
on the prescribed topics. In general, we found that 
instructional time was geared toward building students' 
basic skills. During reading lessons, teachers spent 
about 25 percent of their time on knowledge acquisi- 
tion, which included lecturing or having students read 
(see Figure 5-3a). They spent an additional 25 percent 
of class time having students engage in skills practice. 
These activities included having students answer ba- 
sic questions about a passage they had read or work 
on vocabulaty and reading comprehension exercises. 
Additionally, during reading lessons, teachers spent 
about 10 percent of their time setting up new ac- 
tivities (e.g., providing background information on 
a new topic, giving directions), about one-third of 
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FIGURES 5'3AANO B 

Summer Bridge Teachers Spend about Half of 
Their Time Lecturing and in Skills Practice 

Reading Instructional Categories 




Mathematics Instructional Categories 




M Down/disrupted 


Testing 


■ Housekeeping 


v; Understanding 


. Non-in structlonal set-up 


9 Skills practice 


9 Instructional set-up 


9 Knowledge acquisition 


■ Review 





See the sidebar, "How Teachers Used Their Time in Summer Bridge: 
Definitions of Our Categories" Data are from 2000 Summer Bridge 
observations. 



o 




which included instruction, and 13 percent of their 
time reviewing work that students had completed pre- 
viously either in class or for homework. On average, 
only 8 percent ol Summer Bridge reading lessons were 
devoted to understanding or in-depth student work, 
such as activities that required students to solve prob- 
lems interactively and do more than just practice ba- 
sic skills. 

The proportion of time that teachers spent on in- 
structional activities differed somewhat across math- 
ematics and reading (see Figure 5-3b). In mathematics, 
teachers engaged students in more skills practice and 
spent less time lecturing students and providing in- 
struction (knowledge acquisition). Almost a third (32 
percent) of the time was spent on skills practice in 
mathematics (compared with 25 percent in reading). 
On average, teachers spent only 19 percent of their 
time lecturing in mathematics. As in reading, teachers 
spent 10 percent of their time setting up new activi- 
ties in mathematics lessons. They spent 16 percent of 
their time reviewing previous work (slightly more than 
the 13 percent in reading). Teachers spent even less 
time having students work on in-depth academic ac- 
tivities (understanding) than they had in reading, with 
only 5 percent of instructional time devoted to more 
extensive problem-solving activities. 

How Efficiently Was Time Used 
in Summer Bridge? 

The focus on instruction and skills practice suggests 
that Summer Bridge classrooms were relatively fast- 
paced environments in which teachers were structur- 
ing lessons and student work with little down time 
and quick transitions. Indeed, one hypothesis to ex- 
plain why students felt that they were working harder 
and why we observed large test-score gains in Sum- 
mer Bridge is that the program produces more focused 
instructional time than the school year. Smith (1998) 
found that poor time man^ement and low levels of 
instructional time erode the opportunity to learn in 
Chicago’s elementary schools. 
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In general, there was little time spent on non-in- 
structional activities in Summer Bridge, On average, 
over three-quarters of class time was devoted to either 
providing instruction, having students practice skills, 
or setting up and reviewing assignments and activi- 
ties. Less than a quarter of class time, on average, was 
spent on non-instructional activities. Much of this time 
was spent on the announcements and act iv ides that 
naturally consume some portion of students time in 
school (e.go taking attendance, handing out materi- 
als), A small portion of time was also spent on class- 
room management, discipline, or with some type of 
disruption. 



Oid feadiers terf in How Tliey Groyped 
Slydents for Inslmction? 

The Summer Bridge curriculum, particularly in the 
third and sixth grades, called for teachers to use small 
group instruction very frequently, Importandy, this was 
the one area in which most teachers did not follow the 
curriculum. In both reading and mathematics, the 
average teacher worked with the entire class close to 
two- thirds of the time (see Figures 5-4a and 5 -4b), 
Most of the activities that we observed — lecturing, set- 
ting up assignments, reviewing student work, and 
housekeeping tasks — were conducted in a whole-class 
structure. The rest of class time was usually spent with 
students working on assignments individually (29 per- 
cent in reading and 26 percent in mathematics), most 
often in exercises that emphasized skills practice. In 
both mathematics and reading, teachers used groups, 
on average, only 5 percent of the time, 

Oifferencas across Grades m lime ianagemenl anil 
ftislmctionalAclivilies 

Third-grade teachers differed from their middle grade 
counterparts both in how they grouped students and 
in the amount of time they spent on instructional ac- 
tivities. Third-grade teachers spent almost three-quar- 
ters of Summer Bridge time with students grouped as 



FIGURE 5-4AANDB 

Teachers Spent the Majority of Class Time 
Providing Whole Class Instruction and Very 
Little Time Working with Students in Groups 

Reading Group Codes 




Overall Third grade Sixth grade Eighth grade 



Mathematics Group Codes 





Overall Third grade Sixth grade Eighth grade 

■ Whole class s Individual 

91 Groups or pairs Mixed grouping 

Data are from 2000 Summer Bridae observations. 
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How Teachers Use Time in Summer Bridge: Category Definitions and Examples 

Down/disrupted: Activities that bring instruction or student work to a stop and when the classroom is not engaged 
in any task (e.g., teacher steps out of the room for five minutes while the class waits). 

Example: Neither the teacher nor students are in the room at the beginning of class. The teacher arrives and 
students straggle in one-by-one. A school administrator enters and has a conversation with the teacher about 
students attending a high school orientation while the students are sitting at their desks talking. 

Housekeeping: Necessary but non-instructional activities such as taking attendance, handing out paper and read- 
ing announcements. Also pertains to group maintenance activities such as pep talks or the brief chatting that often 
begins a class. 

Example: An administrator talks to students about a letter sent by the school telling them to attend an orientation on 
a Summer Bridge day. She tells the students that they should not go to the orientation. The administrator leaves and 
instruction resumes. 

Set-up: Instructional set-up includes reviewing instructions, completing an example together as a whole class to 
make steps of an activity clear. Non-instructional set-up includes handing out materials, getting into groups, and 
providing directions. 

Example (Instructional Set-up): The teacher tells students to open their textbooks and look at a chart about main 
ideas and details. She instructs them to describe something they can see from their seats using the chart as a 
guide. They should write several sentences about the object. A student asks for clarification and the teacher de- 
scribes a bookcase as an example of what she wants. 

Review: Reviewing or checking work without much instruction or feedback. Calling out answers to math problems 
or recalling the who, what, when, and where of a story chapter. May include brief episodes of "reteaching," but if the 
reteaching is extensive or evolves into new instruction, this code is not used. 

Example: The teacher explains that the day's instructional focus is determining whether the main idea is stated or 
not. She asks a student to summarize a passage they read about Australia. The student responds that the story was 
about a girl who lived on a sheep ranch and was educated through the mail. 

Knowledge acquisition: Lecturing, learning via listening, not talking or reading, and most teacher-led didactic 



a whole class. In contrast, eighth-grade teachers spent 
only 55 percent of reading class time with students 
grouped as a whole class. Similarly, third-grade teach- 
ers had their students work individually much less of- 
ten than sixth- and eighth-grade teachers. Since most 
skills-related work is completed individually, it is not 
surprising that the whole-class environment created 
by third-grade teachers also resulted in a smaller pro- 
portion of class time being devoted to skill work in 
both reading and mathematics than in the sixth and 
eighth grades. Third-grade teachers spent substantially 



more time lecturing and providing whole-class instruc- 
tion, but less time reviewing previous work. In Chap- 
ter 4, we found that third-grade teachers were also less 
likely to report tailoring the curriculum to meet stu- 
dents’ needs and working with individual students 
outside of class. Third-grade teachers may have either 
lacked the capacity themselves, or felt that their stu- 
dents lacked the capacity to work individually or in 
small groups, despite the emphasis on these activities 
in the curriculum. 
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instruction. Includes checks on basic comprehension (presenting and questioning on who, what, when, and where). 
Watching a video. 

Example: The teacher explains how to find the main idea, "You figure out what the story is about and find details 
to support it. “A student reads aloud about stated and unstated main ideas from the book. The others are to follow 
along in their books. The teacher asks a student to read aloud a passage about migrating birds. The students are 
asked to underline the phrase that gives the main idea. A student reads a segment of the story to the class. The 
teacher asks her what the main idea of her paragraph was, and the student responds. 

Skills practice: Skill and seat work such as fill-in-the-blank exercises, basic skills work in language arts such as 
grammar and vocabulary, math calculations, labeling maps or graphics. 

Example: The teacher asks the students to underline the main idea in paragraphs 7, 4, 5, 6, and 9. Some students 
talk to each other about what the answers might be; others are working on their own to find the answers. 

Understanding: Activities that involve higher-order thinking and/or exploring material in depth. These activities 
require students to produce a substantive piece of work, such as a discussion, an individual or group project, 
writing assignments, design work, graphing, mapping and modeling, hypothesis testing, estimation, analysis, 
synthesis, evaluation, or other forms of problem solving. Activities involve multiple steps and are understood to 
be part of a whole. 

Example: Students work quietly for ten minutes on charts they are creating. When they all finish, the teacher tells 
them to write about the first two chapters of their novels using a chart like the one they Just completed to show the 
main idea and supporting details. As students work, the teacher walks around the class assisting students. 

Taking a test, practice test, or test prep: Taking a test, practice test, or activities that are explicitly linked to test- 
taking strategy. Discussing directions for the IT8S, strategies for answering multiple-choice questions, reading 
strategies that apply only in timed situations, math strategies that rely on having answer choices available, or the 
phrasing of questions as they appear on ITBS. 



Note: No example of test taking/test prep is included here as it was observed very rarely (approximately 7 % of the time). 



Differences across Instructional Environments in 
Instructional Activities and Time Use 

We find litde variation in both grouping and instruc- 
tional activities across our four instructional environ- 
ment categories (tailored, sufficient, minimal, and 
insufficient). This is most likely due to how closely 
teachers followed the daily lesson plans and the 
programs prescribed pacing. Of the differences we do 
find, teachers who provided more quality instruction 
during reading lessons tended to provide more whole- 



class instruction, had students work in groups more 
often, and had students spend less time working indi- 
vidually. A more pronounced difference across the cat- 
egories was that teachers who provided tailored 
instruction differed in the quality and variety of inter- 
actions that they engaged in with their students. These 
teachers provided more feedback to students, both as 
a group and individually, and made a greater effort to 
present the material so that it was accessible to both 
their strongest and their weakest students. Teachers 
who provided tailored instruction went beyond the 
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prescribed topics and activities to create learning 
environments that were geared to the needs of the 
students in their classrooms. 

Did Teaching Matter? A Look at Learning Gains 
across Instructional Environments 

A final question is: Did instructional variation and 
differences in the learning environments of Summer 
Bridge classrooms affect how much students learned 
in the program? The best way to evaluate this ques- 
tion is to link students' learning gains to their Sum- 
mer Bridge teachers. Unfortunately, although we are 
able to link Summer Bridge students to their schools, 
we are unable to link them directly to their teachers in 
schools with more than one Summer Bridge classroom 
in a particular grade. Of our 1 2 schools, about 60 per- 
cent had one class per grade and 40 percent had two 
or more. We can look at whether teacher quality and 
learning gains were correlated, but there is substantial 
error in our method due to the cases where there are 
multiple classrooms within a grade. In those instances, 
students' learning gains are only partially attributable 
to the teacher we observed. Thus, our analysis under- 
estimates the relationship. We looked at the average 
adjusted learning gain in eacli school at each grade 
accounting for the incoming test scores and demo- 
graphic characteristics of students as well as school char- 
acteristics.'* This analysis was conducted for all Summer 
Bridge classrooms in 2000 (see Appendix J). 

We compared the Summer Bridge adjusted reading 
test-score gains in the third, sixth, and eighth grades 
for each school in which we conducted observations. 
Even with the above-mentioned measurement error, 
students in grades where the observed teachers' les- 
sons were rated as providing tailored instruction had, 
on average, substantially larger adjusted test-score in- 
creases in Summer Bridge than the average Summer 
Bridge student (see Figure 5-5). Students in our 12 
schools who were in grades where the observed teacher 
was rated as providing sufficient instruction — the in- 
struction that we observed most frequently in Sum- 
mer Bridge — had slightly higher adjusted test-score 




FIGURE 5-5 

Observed Classrooms with High-Quality 
Instruction Had Larger Adjusted Learning Gains 
than the System Average in 2000 
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On this figure, zero represents the average Summer Bridge adjusted 
learning gain in reading in each grade for the entire system. Scores above 
and below zero indicate learning gains above or below that of the average 
Summer Bridge classroom In 2000. 



increases than the average Summer Bridge student. 
Finally, the students of teachers who pro vided litde or 
no instruction (teachers whose lessons were rated as 
minimal or insufficient instruction) had adjusted test- 
score gains slightly lower than the average Summer 
Bridge student in 2000. 



Summary 



Almost all of the Summer Bridge teachers we observed 
used the daily lesson plans and were able to maintain 
the program's pacing. There was little variation across 
Summer Bridge teachers in the types of activities they 
engaged students in and how they grouped students 
in their classrooms. Rather than differences in instruc- 
tional activities or grouping, the qualities that set well- 
implemented lessons apart in Summer Bridge were the 
connections that teachers made with students, the ex- 
tent to which they provided them with individual sup- 
port and feedback, and teachers’ efforts to provide 
instruction that was both accessible and engaging. In 
both Chapters 4 and 5, we find evidence that some 
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Comparison of Time Use in Summer Bridge to Use in the School Year 



We compared the amount of non-instructional time we 
saw in Summer Bridge to that found by BetsAnn Smith 
in her 1998 Consortium report It's About Time: Oppor- 
tunities to Learn in Chicago's Elementary Schools. In that 
report, effective time use is defined as having less than 
20 percent non-instructional time in the school day. We 
found that teachers in Summer Bridge used their time 
effectively in 54 percent of the lessons we observed. 
During the 1994-96 school years when the data were 
collected for Smith's study, about half (49 percent) of 
teachers used their time effectively. Even though these 
numbers are quite similar, Summer Bridge did have a 
larger proportion of teachers who used their time effec- 
tively. Also, although we found that 25 percent of the 
Summer Bridge lessons we observed lost 10 percent or 
less of class time to non-instructional activities, only 4 
percent of teachers during the 1 994 through 1 996 school 
years used their time that effectively. 



Summer Bridge Teachers Used Their Time Somewhat 
More Effectively Than Teachers In the 1994-96 
School Years 
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teachers tailored instruction to the needs of their stu- 
dents. Here we show evidence that teachers who pro- 
vided individual attention and feedback had a positive 
impact on students’ learning gains. 

In this chapter, we have looked at how instruction 
varies when teachers are provided with a mandatory 
curriculum that includes daily lesson plans. The man- 
datory curriculum does appear to have resulted in simi- 
larities in instructional activities and classroom 



structure. Teachers were told what to teach and what 
activities to engage their students in, and they did so. 
We also find, however, that what teachers themselves 
bring to the classroom still matters. The extent of in- 
dividualized instruction that teachers provided and 
how much they worked to gear their lessons to their 
students’ needs varied extensively across Summer 
Bridge classrooms, despite the mandatory curriculum. 
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A s we saw in Chapter 2, CPS students experienced significant test- 
score increases in Summer Bridge, particularly in the sixth and eighth 
grades. Previous research has found, however, that gains made in 
summer school tend to diminish over time.^ Our findings and previous 
research raise the question: To what extent did Summer Bridge benefit 
students in the long run? In this chapter, we examine this question by 
looking at two-year learning trends among third- and sixth-grade students 
who attended Summer Bridge in 1997 and 1998.^ 

Looking at Sustainability: A Research Approach 

If Summer Bridge learning gains represent improvement in students' test 
scores that can be sustained over time, we would expect students who at- 
tended Summer Bridge to have greater test-score gains on the ITBS ad- 
ministered the following spring than similar students who did not attend. 
Thus, we need a comparison group to examine whether Summer Bridge 
test-score gains are sustained. Since all students whose scores are below the 
cutoffs are required to attend Summer Bridge, there is no comparable group 
of students with identical test scores who do not attend the program. 

How do we obtain an adequate comparison group? One approach would 
be to compare the performance of students in years prior to the CPS pro- 
motional policy (pre-1996) to that of students who attended Summer 
Bridge. The problem with this method is that test scores rose substantially 
after the 1996-97 school year, and supports for low-achieving students, 
such as the Lighthouse after-school program, were added at that same time. 
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Thus, it is possible that Chicago’s low-achieving stu- 
dents may have experienced increased learning gains 
without Summer Bridge. A student who scored a 4.6 
on the ITBS in 1997 might have learned more be- 
tween the end of sixth and the end of seventh grade 
than a similar student prior to the policy, either be- 
cause of Summer Bridge or because she received more 
attention during the school year. As a result, compar- 
ing low- achieving students’ achievement before and 
after the policy would overestimate the effect of 
Summer Bridge by confounding Summer Bridge ef- 
fects with overall improvement in student test scores 
after the 1996-97policy. 

A second approach is to take advantage of the fact 
that there is wide variation from test to test in stu- 
dents’ performance on the ITBS. Because a single test- 
score cutoff was used to determine whether students 
would be promoted or retained, many students who 
attended Summer Bridge had scores that were just 
below the cutoff, while many who did not attend had 
scores just above. For example, sixth graders who scored 
a 5.2 went to Summer Bridge, while sixth graders who 
scored a 5.4 did not. In many cases, this decision re- 
sulted from answering one question right or wrong on 
the reading test. Thus, while the achievement of all 
sixth graders who did not meet the test-score cutoffs 
in spring is not comparable to that of all sixth graders 
who did, we would expect sixth graders who were close 
to the cutoffs on either side to be similar. 

There are analytic problems in using this group as a 
comparison for students who attended Summer Bridge. 
The first problem, as we have discussed throughout 
this report, is regression to the mean. As many stu- 
dents in Summer Bridge may have wound up in the 
program because of a low test score, we might expect 
their following school year to be better than average 
even if the program had no effect. A second problem 
is maturation. Students with low test scores may, par- 
ticularly in the third grade, “mature” and begin to catch 
up with their counterpans. Thus, we might expect that 
even before the policy was implemented, students with 



test scores below the cutoffs would follow poor third- 
or sixth-grade years with better than average fourth- 
or seventh-grade years. 

We address these problems by looking at test- score 
trends for students who have scores within a narrow 
range above and below the cutoffs from both before 
and after the policy was implemented. We start with a 
group of students whose ITBS scores fall within a three- 
month band around the cutoffs — a group of students 
we expect to be similar in terms of achievement.^ For 
example, in the third grade, we compare students 
whose spring ITBS scores were between 2.5 and 2.7 
GEs and who attended Summer Bridge to another 
group of third graders who scored between 2.8 and 
3.0. We call the first group “Summer Bridge high scor- 
ers” and the second group “comparison group.” We 
also constructed two additional comparison groups 
made up of students enrolled in a CPS school before 
the 1996-97 school year. Our “pre-policy Summer 
Bridge-eligible group” comprises students who would 
have been required to attend Summer Bridge if the 
promotional policy had been in effect. Therefore, for 
the tftird grade, students in this group would also have 
scored between 2.5 and 2.7 on the ITBS. Similarly, 
our “pre-policy comparison group” comprises those 
third graders who scored between a 2.8 and a 3.0 on 
the ITBS prior to the policy. We estimate whether Sum- 
mer Bridge gains are sustained by comparing the dif- 
ference in learning gains between Summer Bridge high 
scorers and the comparison group to the difference in 
learning gains between the two pre-policy groups with 
the same test scores. 

This analysis includes students who were required 
to attend Summer Bridge for reading only. It also ex- 
cludes retained students in order to distinguish be- 
tween the effect of Summer Bridge and retention.'^ 
Summer Bridge high scorers make up approximately 
40 percent of third graders and 35 percent of sixth 
graders who were required to attend the program and 
were tested at the end of the summer. Students who 
attended Summer Bridge because of their mathemat- 
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Using Test Scores to Construct Comparison Groups 

Missing one question on the Iowa Tests of Basic Skills (ITBS) can mean the difference between meeting the 
promotional cutoff and attending Summer Bridge, For example, on the 1998 reading ITBS, third graders who 
had 14 questions correct received a 2.7 while students an- 
swering 1 5 questions correctly received a 2.8. The cutoff 
score for third graders was 2.8. 

When we look at the distribution of test scores in 1998, we 
find that approximately 14 percent of third-grade students 
scored between 2,5 and 2.7 (raw scores of 12 to 14 correct 
out of 36 questions). This is our Summer Bridge high scorer 
group. Another 11 percent of third graders scored between 
2.8 and 3.0 (raw scores of 1 5 to 1 7). The latter group is our comparison group. In addition, we constructed two 
additional comparison groups who attended a Chicago public school prior to 1996, or before the promotional 
policy was in place. Sample sizes across our four groups in the pre- and post-policy years are in the table above. 
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ics score, because of low attendance, or other poor 
school year performance, were excluded from the com- 
parison group. 

In this chapter, we look at the performance of third 
and sixth graders who attended Summer Bridge in 
1997 and 1998 in order to look at test-score increases 
one and two years after Summer Bridge attendance. 
Third graders had moderate adjusted gains in Sum- 
mer Bridge, approximately one to two months. As dis- 
cussed in Chapter 2, third graders at the lowest risk of 
retention had slightly smaller gains in Summer Bridge 
than third graders at the highest risk. Thus, for third 
grade, our estimate of sustainability coidd be consid- 
ered conservative. Sixth graders in Summer Bridge also 
had moderate adjusted test-score increases in reading 
of approximately four months. Adjusted test-score in- 
creases were higher for sixth graders at low risk of re- 
tention. Thus, our estimate of sustainability in sixth 
grade is probably a best-case scenario because we are 
looking at students who likely had die largest gains 
after Summer Bridge. 

In Chapter 2, we found that eighth-grade students 
had the largest test-score gains in Summer Bridge. 
Unfortunately, we are unable to investigate whether 



these gains are sustained because CPS ninth graders 
take a different standardized test, the Tests of Achieve- 
ment and Proficiency (TAP). TAP and ITBS scores 
are not directly comparable. In future research, we will 
investigate the long-term implications of eighth-grade 
Summer Bridge effects by examining students’ high 
school performance and dropout rates. 

Oeparting from Grade Equivalents to 
Study Growth over lime 

Throughout this report, we have reported Summer 
Bridge adjusted test-score gains in the GE metric, 
which the school system uses to make promotional 
decisions. Because it is measured relative to an average 
student in a certain grade, the GE metric is adequate 
for looking at test-score gains within a grade, but it is 
inadequate for examining growth over time and across 
groups of students. Also, because different forms of 
the ITBS are administered each year, the GE metric 
malces comparing scores across years difficult.^ To ad- 
dress these problems, the Consortium conducted an 
extensive equating study that converted ITBS test 
scores to a logit metric using Rasch models that are 
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Generalizing Our Findings on Sustainability 



This chapter limits analysis of 
sustainability to students who 
were just below and above the 
Summer Bridge test-score cutoff 
in reading because, as we have 
argued, these students can be 
considered an adequate compari- 
son group — students in these 
groups have scores in a range 
where we expect that a good or 
bad testing day could have re- 
versed whether or not they met the 
cutoffs. Our results, however, can 
only be generalized to high-scor- 
ing Summer Bridge students; we 
do not know whether the 
sustainability of Summer Bridge 
gains would differ for students 
with lower skills. As we saw in 
Chapter 2, third graders with the 
lowest reading skills had the 
largest test-score gains in Sum- 
mer Bridge. 

There are two reasons that we did 
not include an analysis of test- 
score gains among students with 
lower test scores. First, as de- 
scribed in Chapter 2 (see sidebar, 
page 43), over a quarter of third 
graders in Summer Bridge had 
test scores at or below the level 
of chance on the Iowa Tests of 
Basic Skills (ITBS), making analy- 
sis of their long-term learning 
gains problematic. In addition, 
many students who had pre-Sum- 
mer Bridge test scores below 2.5 
were retained after the summer. 
This means we would be compar- 
ing students who had similarly low 
scores but those in the post-policy 



group would have been retained while 
those from before the policy would 
not have been. 

In order to separate the effects of re- 
tention from the effects of Summer 
Bridge, we chose to limit our analy- 
sis to high-scoring students, most of 
whom were promoted either because 
they met the test-score cutoff after 
Summer Bridge or because their test 
scores were close enough to be pro- 
moted by alternate criteria. Although 
we excluded the small proportion of 
high-scoring students who were re- 
tained from this analysis, we did con- 
duct the analysis with retained 
students and found similar results. As 
an additional check, we also exam- 
ined the one-year sustainability of 
Summer Bridge results in 2000. As 
described in Chapter 1, in 2000, the 
promotion policy was altered to in- 
clude a range around the test-score 
cutoffs. Thus, all of the students in 
our high-scoring group were pro- 
moted whether or not they met the 
cutoffs at the end of the summer. 
One-year effects for 2000 are quite 
comparable to those found in 1997 
and 1998, confirming that our results 
are not driven by the exclusion of re- 
tained students from the analysis. 

Our results differ slightly from an 
analysis of Summer Bridge 
sustainability conducted by Brian 
Jacob and Lars Lefgren. Jacob and 
Lefgren used the "discontinuity" cre- 
ated by the promotional test-score 
cutoffs to estimate a two-stage model 
that predicted the probability of at- 



tending Summer Bridge. They then 
estimated the effect of the program 
on achievement for students who 
were within one standard deviation 
of the cutoff on the basis of the 
probability of attending Summer 
Bridge as well as prior test scores. 
They found that Summer Bridge in- 
creased the subsequent test perfor- 
mance of third graders but not sixth 
graders. In essence, their findings 
were similar to ours in that they 
found that learning gains for low- 
achieving third graders who at- 
tended Summer Bridge were higher 
two years later than gains for stu- 
dents above the cutoff who did not 
attend Summer Bridge. They did 
not find evidence for sustainability 
in the sixth grade, however. This 
difference in our findings most 
likely occurs because Jacob and 
Lefgren looked at students only in 
1 997 and 1 998 and did not include 
a pre-policy comparison. As seen 
in Figure 6-5, two-year learning 
gains post Summer Bridge were 
smaller for sixth graders. In the 
absence of the policy we would 
have expected the gap to widen 
slightly: however, the pre-policy 
comparison suggests that third- 
grade students Just below the cut- 
off might have had greater 
test-score gains regardless of Sum- 
mer Bridge. Thus, adding the pre- 
policy comparison increases our 
estimates of Summer Bridge 
sustainability for the sixth grade 
but decreases estimates for the 
third grade. 
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comparable across time and within and across levels.^ 
In this chapter, we report restdts in this Rasch metric. 
The analysis was also conducted using GEs and re- 
sults were comparable. 

Test-Score Trends in 1997 and 1998 for Summer 
Bridge High Scorers and the Comparison Group 

The assumption behind our analysis is that students 
who score slightly above the cutofF are similar to those 
who score slightly below, and that the primary differ- 
ence between them is that one group attended Sum- 
mer Bridge and the other did not. 

In the third and sixth grades in 1 997 and 1998, the 
demographic characteristics of these two groups were 
roughly similar (see Figure 6-1). Although there were 
slightly more African-American students in the Sum- 
mer Bridge group than in the comparison group, the 
difference wa 5 small. In both the GE and Rasch metrics 
(logits), the prior year (second and fifth grade) test 
scores of the comparison group were slightly higher 
than those of the Summer Bridge high scorers. In the 
GE metric, students who attended Summer Bridge and 
scored just below the cutoff had prior year (second 
and fifth grade) test scores that were, on average, two 
months below students in the comparison group. Thus, 
test scores for Summer Bridge high scorers were slightly 



FIGURE 6-1 

Summer Bridge High Scorers and Students in the 
Comparison Group Have Similar Demographics 
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below the comparison group both in the testing year 
and in the prior year 

Figures 6-2a and 6-2 b show test-score trends for 
both Summer Bridge high scorers and the compari- 
son group, charting growth from one year prior to the 
promotional gate grade (fifth grade) through two 
years after Summer Bridge participation (seventh 
and eighth grades). 

The data indicate that students in the higher-scor- 
ing Summer Bridge group followed the program with 
relatively higher ITBS scores, narrowing the gap be- 
tween themselves and the comparison group. At the 
same time, we also see that Summer Bridge students 
tended to have very poor sixth-grade years, and that 
the gap between Summer Bridge high scorers and the 
comparison group was wider in the sixth grade than 
in the fifth. One interpretation of the pattern of the 
poor sixth grade followed by a good seventh grade is 
that Summer Bridge allowed those students who had 
a bad year to catch up and close the gap between them- 
selves and their classmates. Another interpretation is 
that many students are required to attend Summer 
Bridge because of bad test days (and, therefore, scores 
that underestimate their true ability) and post-Sum- 
mer Bridge learning gains are inflated due to a poor 
sixth-grade score followed by a better seventh-grade 
score. If we compare only pre-(fifth grade) and post- 
(eighth grade) test performance for the two groups, 
we see that students in the Summer Bridge high scor- 
ing group had slightly higher average learning gains 
than the comparison group (0.03 logits in 1997 and 
0.04 logits in 1998, or about 7 percent of the average 
learning gains from sixth to seventh grades, see Ap- 
pendix K). 

The trend for third grade is very similar. Third grad- 
ers in the Summer Bridge high scorers group were be- 
low the comparison group in second grade and that 
gap widened during third grade (see Figures 6-3a and 
6-3b). Students who attended Summer Bridge, how- 
ever, tended to have better post-Summer Bridge years 
and maintained these gains so that by the fourth and 
fifth grades they closed the gap between themselves 
and the comparison group. 



O 

ERIC 



104 



BEST COPY AVAILABLE 



104 ■ CHARTING REFORM IN CHICAGO 



FIGURES 6-2A AND B 

Trends in ITBS Reading Scores of 1997 
Sixth-Grade Summer Bridge High Scorers 
and 1997 Comparison Group 
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FIGURES 6-3A AND B 

Trends in ITBS Reading Scores of 1997 
Third-Grade Summer Bridge High Scorers 
and 1997 Comparison Group 
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Trends in ITBS Reading Scores of 1998 
Sixth-Grade Summer Bridge High Scorers 
and 1998 Comparison Group 
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Trends in ITBS Reading Scores of 1998 
Third-Grade Summer Bridge High Scorers 
and 1998 Comparison Group 
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Trends in Pre-Summer Bridge Years 

How much of the test-score improvement among stu- 
dents in the Summer Bridge high scorer group is due 
to the effect of the program, and how much is attrib- 
utable to regression to the mean that may have oc- 
curred regardless of participation? We gain some insight 
if we look at groups of similar students from before 
Summer Bridge was implemented. If the narrowing 
of the gap that we observe is simply the result of matu- 
ration or regression to the mean, we would expect to 
see similar trends for students with test scores just above 
and below the cutoff before CPS initiated its efforts to 
end social promotion in the 1996-97 school year. The 
comparison groups comprise third and sixth graders 
in 1993 and 1994. Because these students were in the 
key grades during predominantly non-policy years, 
they provide an adequate comparison group for what 
might occur in the absence of the policy." 

As in our previous analysis, we looked at four-year 
learning trends among students who were in the third 
and sixth grades in 1993 and 1994 and whose test 
scores fell slighdy above and below the cutoff (see Fig- 
ures 6-4a and 6-4b). Our pre-policy Summer Bridge- 
eligible group is those students who scored just below 
the promotional gate cutoff but did not attend Sum- 



Trends in ITBS Reading Scores of 1993 
Third-Grade Pre-Policy Summer Bridge-Eligible 
Group and 1993 Comparison Group 
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parison group is those students who scored slightly 
above the test-score cutoff in both years. 

Comparing the pre-policy groups trends to those 
of the post-policy group gives us insight into the ex- 
tent to which the gains made by Summer Bridge stu- 
dents can be attributed to regression to the mean. If 
Summer Bridge gains were due to regression to the 
mean, we would expect that the gap between the two 
pre-policy groups would narrow as the gap between 
the post-policy groups did. When we look at the gains 
for the first year after sixth grade only, regression to 
the mean looks like a plausible explanation, as the pre- 
policy Summer Bridge-eligible group narrowed the gap 
to the comparison group, which is similar to what we 
saw with the post-policy groups. However, if we ex- 
amine more long-term trends, from fifth to eighth 
grades, we see that the gap between the 1 993 pre-policy 
Summer Bridge-eligible group and the comparison 
group widens very slightly instead of narrows. In 
addition, in contrast to the post-policy groups, the 
two groups show almost identical gains (Rasch scores 
of 1.47 for the Summer Bridge-eligible group and 
1.48 for the comparison group). We see similar 
trends when we compare our pre- and post-policy 

FIGURES 6-4AAND B 

Trends in ITBS Reading Scores of 1993 
Sixth-Grade Pre-Policy Summer Bridge-Eligible 
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Note: The pre-policy Summer Bridge-eligible group consists of those students in 1993 who would have had to attend Summer Bridge if the policy had 
been In place. Trends were similar for 1994. 
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third-grade groups. Unlike the post-policy groups, 
which converge, the two pre-policy groups have simi- 
lar gains and a slight widening of the gap. Because the 
pre- and post-policy groups show different patterns, it 
appears that the gains of the Summer Bridge group 
are not due to regression to the mean. 

Comparing Trends before and after 1996: 

Are Summer Bridge Gains Sustainable? 

When we look at test-score, trends for students one 
year prior to and then two years after third grade, we 
find that students who attended Summer Bridge had 
slightly larger average ITBS gains in the two years af- 
ter Summer Bridge than students who were just above 
the cutoff. We also see that the difference between the 
four-year learning gains of students just 
below and above the cutoff was a little 
bit larger after 1 996, when students at- 
tended Summer Bridge, than before. 

For example, the 1998 third graders in 
our Summer Bridge high-scoring 
group had an average gain in the two 
years after Summer Bridge of 0.55 in 
the Rasch metric compared to 0.47 for 
the comparison group. Average learn- 
ing gains for these students were 17 
percent higher than those for the com- 
parison group. If we look at these stu- 
dents’ four-year learning gains (second 
to fifth grade), which account for the 
fact that Summer Bridge students had 
very poor third-grade gains, we find 
that the four-year learning gains are 
slightly higher for the Summer Bridge 
high scorers (see Figure 6-5). 

Although positive, these findings on 
the sustainability of Summer Bridge 
gains are modest. First, we do not find 
evidence that vSummer Bridge high 
scorers did better than students with 
slightly higher test scores who did not 
attend the program. At best, it appears 



that Summer Bridge allowed these students to narrow 
the gap between themselves and students slightly above 
the cutoff. Using the 1993 and 1994 groups as a com- 
parison for what we would expect in the absence of 
Summer Bridge, we estimate that sixth graders with 
test scores just below the cutoff would have learning 
gains approximately 0.02 logits lower than students 
just above the cutoff between the end of fifth and the 
end of eighth grade. In 1998, sixth graders who at- 
tended Summer Bridge and whose reading scores were 
just below the cutoff increased their test scores by 0.04 
logits or about 7 percent of an average one-year learn- 
ing gain more than the comparison group. Taking the 
average of the pre- and post- policy effects, we estimate 
that Summer Bridge students outpaced similar students 
by approximately 0.05 logits. This translates into an 



FIGURE 6-5 



After Four Years, There iSvModest Evidence that 
Summer Bridge Gains Are Sustained 

Differences in Learning Gains of Students Just Above and Just Below 
Promotional Cutoffs in pre-and post-Summer Bridge Groups 
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comparison groups 
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The difference in the pre-policy four-year gains are an average of 1993 and 1994. The difference In 
the post-policy four-year gains are an average of 1997 and 1998. Four-year gains are ITBS 
reading gains (logits). See Appendix K. 
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1 1 percent increase in average one-year learning gains 
for these students.^ 

What Have We Learned about Sustainability? 

In this chapter, we looked at whether students at- 
tending the Summer Bridge program did better in the 
following years than students with similar test scores 
who did not participate. We defined sustainability as 
maintaining Summer Bridge learning gains on future 
ITBS exams. We found some evidence for this type of 
sustainability, although the benefits are modest. Third 
graders who attended Summer Bridge had larger read- 
ing gains between second and fifth grade than stu- 
dents who scored slightly above the cutoff, even though 
their adjusted Summer Bridge test-score g^ns were 
small. Our sixth- grade Summer Bridge group, which 
had large adjusted Summer Bridge test-score gains, did 
appear to have larger test-score gains between fifth and 
eighth grade than students in the comparison group. 
Still, despite the larger test-score gains, students in the 
Summer Bridge group continued to lag slightly be- 
hind their peers with similar scores who did not have 
the benefit of extra instruction in Summer Bridge. 

Taking the average of the pre- (1993 and 1994) 
and post- (1997 and 1998) policy effects, we estimate 



that Summer Bridge students improved their learning 
gains, but they did not accelerate their rate of learning 
in subsequent years. It appears that Summer Bridge 
provided a one-time boost that allowed these students 
to harrow the gap between themselves and other low- 
performing students but did not substantially change 
their performance in school. Our analysis of 
sustainability was limited to the highest scoring Sum- 
mer Bridge students, however, and the program may 
have had a different effect on lower-achieving students. 

In this chapter, we addressed concerns that the posi- 
tive effect of the Summer Bridge program on test-score 
gains would disappear over time. We found evidence 
for the sustainability of these gains, although the ef- 
fect was limited to a modest one-time boost in ITBS 
scores. One interpretation of the results from this chap- 
ter is that students in Summer Bridge learned to take 
the ITBS and did not forget these skills the next time 
they took the test. Another interpretation is that Sum- 
mer Bridge provided a critical intervention for stu- 
dents who were falling behind and allowed them to 
catch up to their classmates with similar achievement. 
In any case, the long-term benefits of Summer Bridge, 
at least in terms of test-score gains, are modest. 
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INTERPRETIVE SUMMARY 



n this report, we have attempted to take a multi-faceted and rigorous 
look at Chicago’s Summer Bridge program. Our interpretive sumniar}^ 
focuses on two primary questions: First, what did we learn about the 
potential role of summer programs in supporting students under high- 
stakes testing? And, second, what do the results of Summer Bridge tell us 
more generally about addressing the needs of low-achieving students? 

The Effects of Summer Bridge 

The goal of Summer Bridge is to provide students with extra support and 
a second chance to meet CPS test-score cutoffs for promotion. Summer 
Bridge has been effective in the short run in producing test-score gains, 
particularly among sixth and eighth graders, and in allowing more stu- 
dents to meet the cutoffs. Our results suggest that summer programs may 
be a promising approach to providing students with extra instructional 
time and remedial support to meet the demands of high-stakes testing. 
Sixth and eighth graders experienced substantial test-score gains when they 
were retested at the end of the relatively brief program. In the third, sixth, 
and eighth grades, the rate at which Summer Bridge students increased 
their test scores was above that experienced by the same students during 
the regular school year. 

We found little evidence to support one of the main concerns about the 
use of summer programs for students under high-stakes testing: namely 
that such programs will produce benefits only for students who are close 
to the cutoffs. Third-grade students who were at the highest risk of failure 
benefited the most from Summer Bridge. Sixth and eighth graders at the 
highest risk had large test-score gains, although these were not as large as 
those of students at more moderate risk. Indeed, one of the most positive 
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One of the most important con- 
clusions of our analysis is that 
low-achieving students re- 
sponded positively to an oppor- 
tunity for summer support. 

findings in this report is that Summer Bridge produced 
relatively uniform gains across demographic and 
achievement groups. 

Students were extremely positive about their expe- 
riences in Summer Bridge. Sixth and eighth graders 
portrayed their Summer Bridge classrooms as environ- 
ments where they were expected to work hard. They 
also reported that their teachers were supportive. Most 
importantly, we found that students who attended 
Summer Bridge were significantly more positive in the 
summer than in the school year about the academic 
environment of their classrooms and the attention they 
received from teachers. The program also had high at- 
tendance rates. Thus, students not only had test^score 
gains in Summer Bridge, they also experienced the 
program as a more positive educational environment 
than their school year classrooms. One of the most 
important conclusions of our analysis is that low- 
achieving students responded positively to an oppor- 
tunity for summer support. 

Part of students perceptions of the program reflect 
the fact that teachers reported providing high levels of 
support and working to adapt the curriculum to stu- 
dents’ needs. Our analysis of instruction suggests that 
Summer Bridge classrooms provided more focused in- 
structional time than during the school year. Thus, 
analysis and teacher reports regarding their own be- 
havior confirmed that students were, in fact, receiving 
more personalized attention and greater academic press 
in the program. 



I’here is also evidence that Summer Bridge students 
had slightly larger learning gains over two years than 
students who were close to the cutoffs and did not 
attend the program. While these results are encourag- 
ing, the effects were small. Summer Bridge did not 
substantially alter low- achieving students’ school year 
learning rates. It appeared to help keep these students 
on track, but did not change their learning trajecto- 
ries. The program provided a short term intervention 
that allowed low-achieving students to raise their test 
scores and seemed to provide an extra boost that kept, 
them from falling further behind. 

These results are quite promising. They address sev- 
eral issues that Chicago or other school systems must 
consider when designing summer programs under 
high-stakes testing. 

Issue 1 : Administrators of summer programs need to 

be concerned about variation in school effects. 

We found that higher achieving schools in Chicago 
ran more effective programs. Students who attended 
Summer Bridge in schools with higher school year 
achievement had larger test-score gains than students 
in lower performing schools. These differences were 
most pronounced in the third grade. Teachers in higher 
performing schools were more positive about the learn- 
ing environment in Summer Bridge and reported more 
attention to the individual needs of students. These 
practices were associated with larger adjusted test-score 
increases. Thus, we do not find that Summer Bridge, 
even with its mandatory curricula and uniform mate- 
rials, ameliorated differences in school quality. How 
can we explain the more positive practices and teacher 
assessments of Summer Bridge in higher performing 
schools? Even within this highly centralized program, 
schools continue to have wide flexibility in. making 
staffing decisions and in other areas of implementa- 
tion. Schools with more effective instructional envi- 
ronments or more highly qualified staff may simply 
be better equipped to run effective summer programs. 
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Issue 2: Summer program administrators need to 
address staffing and staff development issues. 

Our analysis of instruction and teacher practice in 
Summer Bridge suggests that the Summer Bridge cur- 
riculum did, in fact, contribute to consistency in the 
topics that were covered, pacing, and the skill focus in 
classrooms. For the most part, teachers followed the 
curriculum. This seemed to play a critical role in en- 
suring exposure to similar content across classrooms. 
We still found differences across classrooms in instruc- 
tion, however. A key finding in this area is that whether 
teachers knew their students before Summer Bridge 
was an important predictor of test-score increases, stu- 
dent perceptions of support in the program, and 
teacher practice. Teachers who knew a large propor- 
tion of their Summer Bridge students beforehand were 
more likely to report adapting the curriculum to meet 
students' needs and to work more closely with stu- 
dents outside of class time. These effects were most 
pronounced for older students. These results suggest 
that summer programs may be more effective when 
teachers know their students, are familiar with their 
learning styles and behaviors, and are able to extend 
school year relationships into the summer. It may be 
critical, particularly for low-achieving adolescents, to 
have teachers who know them and can motivate them 
while addressing their individual learning needs. In 
high-stakes testing environments, strong connections 
between students and teachers may also serve as a 
motivating factor for teachers. 

Our observations of instruction showed that the 
quality of interactions between teachers and students 
was a distinguishing factor between the most effective 
Summer Bridge classrooms and the average classroom. 
We also found an association between the quality of 
instruction in Summer Bridge and students test-score 
gains. A relatively small proportion of teachers taught 
in ways that engaged students, provided substantive 
feedback, and worked to address individual learning 
needs. We foimd that teachers who provided this tai- 
lored instruction did not differ from other teachers in 



how they followed the curriculum, the skills they em- 
phasized, or the way they grouped students for instruc- 
tion. Rather, differences in the instructional 
environments in Summer Bridge classrooms reflected 
the quality of interactions between students and teach- 
ers — not activities that could easily be added to next 
years curriculum. 

Variation in teacher quality at the low end was also 
important. Despite the curriculum and small class sizes, 
teachers who lacked classroom management skills, 
content knowledge, teaching expertise, and a commit- 
ment to their students simply did not deliver. Poor 
teaching is never acceptable, but in a high-stakes test- 
ing environment with short program duration, it truly 
sells students short. We observed low-quality teaching 
in about 20 to 25 percent of classrooms. Program 
leaders need to monitor classrooms early and inter- 
vene when there is evidence of low-quality teaching. 

Finally, teachers in Summer Bridge followed the 
daily lessons but did not implement the curriculums 
emphasis on using a variety of student grouping con- 
figurations for instruction and activities. Despite the 
small classes and despite the fact that many teachers 
reported that they were attempting to meet individual 
student needs, most Summer Bridge teachers relied 
on traditional, whole-class instruction and individual 
skills practice. This suggests that summer programs 
need to provide extensive staff training and support if 
teachers are to move away from traditional whole- 
class instruction to garner all of the benefits of the 
small classes. 

Issue 3: The effectiveness of Summer Bridge differs 
when gains and promotion rates are compared. 

The goal of a remedial summer school program is to 
improve student achievement. In a high-stakes testing 
environment, however, expectations become more 
complicated — if students' test scores are increasing but 
they fail to meet the test-score cutoffs, is the program 
a success or a failure? We found that Summer Bridge 
students had substantial test-score gains, regardless of 
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how far below the cutoffs they were when they en- 
tered the program. However, the students who were 
furthest behind at the start of the program were the 
least likely to meet the cutoffs, even with very large 
Summer Bridge test-score gains. Is it fair to require 
students to attend a summer program and work hard 
when promotion to the next grade may be virtually 
unobtainable for those 
with the lowest skills? 

Promodon did seem to be 
out of reach for Chicago’s 
lowest skilled students 
until 2000, when the 
school system expanded 
the promotional criteria. 

On the other hand, if stu- 
dents know that their test 
scores are not important 
(because they will be pro- 
moted regardless), they 
may be less motivated to 
work hard. Thus, summer 
program policies need to 
be structured to balance 
the incentive to work 
hard with the possibility 
of success. 

Issue 4: Summer Bridge produced short-term test- 
score gains but did not significantly address the 
ongoing learning problems of low-achieving students. 

Students who participated in Summer Bridge had 
slightly larger learning gains two years after the pro- 
gram than students just above the test-score cutoffs 
who did not participate. However, Summer Bridge did 
not change these students’ learning trajectories, nor 
did it address the fact that they continued to have low 
skills compared to other students. Summer Bridge does 
not change students experiences during the school year. 
Indeed, our analysis suggests that part of the reason 
why students did well in Summer Bridge was because 



it contrasted dramatically with their school year expe- 
riences. Not surprisingly, when Summer Bridge stu- 
dents return to regular school year environments, they 
appear to return to their previous learning rates. Thus, 
although summer programs may be a useful approach 
for supporting students under high-stakes testing, there 
is less evidence that one year of Summer Bridge is an 

effective means for address- 
ing the long-term learning 
needs of low-achieving stu- 
dents. A question that 
arises is whether these stu- 
dents could benefit from 
multiple years of partici- 
pation in remedial sum- 
mer school. 

Issue 5; Summer Bridge 
effects differ across 
grades. 

Specifically, third-grade re- 
sults differed from those 
for the sixth and eighth 
grades. Adjusted test-score 
increases were smaller in 
the third grade. Teachers in 
the third grade were less positive about the curricu- 
lum and seemed to struggle with how to adapt it to 
meet students’ needs. I’hird-grade teachers were also 
less likely to work individually with students and to 
provide support both in and outside of the classroom . 
They also knew fewer of their students before Sum- 
mer Bridge. 

Throughout our work in evaluating Chicago’s ef- 
forts to end social promotion, we have consistently 
found less positive results in the third grade. Third 
graders have the highest rates of retention. Although 
there is evidence that test scores increased following 
the institution of high-stakes testing, eflFects were sig- 
nificantly smaller in the third grade. ^ Third-grade re- 
sults in Summer Bridge mirror our finding that 



. . . our analysis suggests that 
part of the reason why students 
did well in Summer Bridge was 
because it contrasted dramati- 
cally with their school year ex- 
periences. Not surprisingly, 
when Summer Bridge students 
return to regular school year 
environments, they appear to 
return to their previous learn- 
ing rates. 
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high-stakes testing is less effective for younger students. 
We do not know whether this finding, reflects the im- 
portance of motivation and the threat of retention in 
shaping student and teacher behavior, or a mismatch 
between Chicago’s reliance on the ITBS and the needs 
of low-achieving younger students. I hird graders may 
be less likely to respond to high-stakes testing with 
efforts that lead to im- 
proved achievement. It 
may also be more difficult 
for teachers to address 
young students’ substan- 
tial reading and math- 
ematics problems in ways 
that result in test-score 
increases in a short period 
of time. 

It is possible that 
Chicago’s exclusive use of the ITBS leads to a narrow 
focus on reading comprehension that may not address 
the more basic reading skills deficits of low-achieving 
third graders. Perhaps only with a substantial altering 
of the assessments used, teacher training, the curricu- 
lum, and a shift in focus to working on diagnosing 
early reading problems could summer programs be 
more effective for these students. 

How Can We Generalize? 

To what extent is Summer Bridge replicable in other 
school systems? Specifically, what programmatic com- 
ponents (such as small class size, a curriculum aligned 
with the ITBS, the mandatory high-stakes approach) 
contribute to test-score gains, teacher behavior, and 
student experiences? Although our analysis addresses 
these issues, it does not resolve them. There are four 
critical components of Summer Bridge that may shape 
its impact. 

1. How important is class size? 

The reduced class size of Summer Bridge is one of the 
most expensive components of the program. It is also 



an aspect of Summer Bridge that is most supported in 
prior research. Would a similar program with larger 
class sizes still give students the sense that they were 
receiving personalized support? How critical was the 
reduced class size both in recruiting teachers for the 
program and allowing them to work more closely with 
students, particularly given the need to provide a lot 

of individualized atten- 
tion? Because class sizes in 
Summer Bridge were so 
small, there was not 
enough variation within 
the program to answer 
these questipns. 

2. How critical is the 

curriculum? 

We identified two ways that a standardized curricu- 
lum could shape both adjusted learning gains and in- 
struction in Summer Bridge. First, the curriculum and 
its high alignment with the ITBS may be an impor- 
tant factor in producing large test-score gains in the 
program. On the one hand, critics of high-stakes test- 
ing would argue that this high alignment reflects 
“teaching to the test” and would not lead to learning 
gains chat are general izable to ocher assessments. On 
the other hand, these results may be taken as evidence 
that summer programs with clearly defined instruc- 
tional objectives and assessments that measure the skills 
taught in the program are effective. In high-stakes test- 
ing enviromnencs, such alignment may be critical to 
providing all students the opportunity to succeed. It 
makes no sense to hold students accountable for skills 
that have not been taught. A carefully chosen assess- 
ment, decisions about what skills to address, and the 
use of a curriculum and materials that focus on those 
skills should be central concerns in designing summer 
programs under high-stakes testing. 

A second way that the curriculum may be impor- 
tant in shaping the effects of Summer Bridge is in pro- 
viding an incenti ve for teachers to teach in the program 



Third-^ade results in Summer 
Bridge mirror our finding that 
high-stakes testing is less effective 
fior younger students. 
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Rather than viewing the curricu- 
lum as a constraint, Summer 
Bridge teachers were positive 
about the content and the mate- 
rials they were provided. 



and the supports they need to help low- achieving stu- 
dents, Teachers relied heavily on the curriculum, fol- 
lowing the daily lesson plans and keeping up with the 
prescribed pace. Rather than viewing the curriculum 
as a constraint, Summer Bridge teachers were positive 
about the content and the materials they were pro- 
vided. We do not know what effect running such a 
program without a prescribed curriculum would have 
on recruitment of teachers or on their ability to pro- 
vide instruction. 

3. Is the effectiveness of Summer Bridge the result of 
high-stakes testing? 

The high-stakes testing environment can be thought 
of as a programmatic component of Summer Bridge. 
Throughout this report, we have highlighted the idea 
that student motivation may play a role in the magni- 
tude of gains we observe in the program. Motivational 
responses that result in students exerting extra effort 
on the day of the test would lead to overestimates of 
the programs impact. However, motivational responses 
that result in students working harder in school 
throughout the summer suggest that high -stakes test- 
ing environments may be critical in shaping summer 
program effects. Would we have observed these gains 
if students were not required to attend? Would we have 
observed them if students did not face the threat of 
retention? Finally, would teacher behavior change if 
students did not face such high stakes? 

As the new federal legislation on testing takes hold, 
many school districts will be under pressure to raise 
test scores among low-achieving students. Our find- 



ings on the effects of Summer Bridge should not be 
taken as evidence that summer programs are a way to 
raise test scores when stakes are established for the 
school, rather than for the student. 

4. How critical is administrative and fiscal support? 

While our analysis provides a very comprehensive 
evaluation of Summer Bridge, it is a look at only one 
program embedded in a very specific policy initiative. 
Part of the context of Summer Bridge was the signifi- 
cant administrative and financial support given to the 
program. Chicago’s effort to end social promotion was 
the administration s central policy focus, and Summer 
Bridge was seen as a core component of that initiative. 
The initiatives high profile garnered substantial ad- 
ministrative attention for Summer Bridge and led to 
the provision of significant resources for improving 
the test scores of the lowest performing students. Sum- 
mer Bridge is also embedded in a larger accountability 
program that led to a general focus on improving test 
scores. Thus, part of the success of Summer Bridge 
may be dependent upon its perceived importance on the 
parr of administrators, schools, and teachers. 

Policy Tradeoffs 

I’his report examines the potential for summer school 
to provide short-term support for students. Policy de- 
cisions in large school systems always require choos- 
ing among an array of options. Summer programs are 
costly and take resources away from activities such as 
professional development that often occur during that 
time. We do not analyze the cost effectiveness of Sum- 
mer Bridge. This would require comparing the pro- 
gram to alternative approaches to raising achievement 
through such things as early intervention, pre-school, 
and professional development, or through school year 
investments in curricula, materials, and class size re- 
ductions for low-skilled students. 

We find that although summer programs may be a 
useftil intervention for students who are behind, they 
are not a substitute for effective instruction during the 
school year. There is no evidence that Summer Bridge 




115 



BEST COPY AVAILABLE 



RESULTS FROM SUMMER BRIDGE 



115 



affected school year learning rates. The caveat to this 
conclusion is that we do not know if teachers’ experi- 
ences in Summer Bridge led to changes in their school 
year practice. Summer Bridge was not designed ex- 
plicitly to lead to changes in school year instruction, 
although this may be a promising route to take in the 
future. What we have learned is that summer can pro- 
vide an opportunity for teachers to work closely with 



students in an environment that is different from the 
school year and can benefit students who are in need 
of extra support, A promising next step might be to 
increase instructional innovation and professional de- 
velopment in Slimmer Bridge so that teachers are ex- 
posed to new methods that could benefit their efforts 
during the school year. 
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Appendix A 



Instructional Foci from Summer Bridge Curriculum 
Sixth-Grade Reading 



Instructional Foci from Summer Bridge Curriculum 

Third-Grade Mathematics 



Percent of Questions 
on ITBS 



Percent Time Listed 
as Instructional Focus 



100-1 



90- 



80- 



70- 



60- 



50- 



40- 



30- 



20 - 



10 - 



o-h- 



Undofstand factual information 






Ojw< ficnt^slwiv. 
Intvencas. 



infer traits, footings, or 
motivalions of characters 



Apply, depict, or predict outcome 
Determine main idee ot' topic 



Recogntts aspects of mood, 
iooe. style, or etruclure ' 



inicrprcl non>titoral language 



Understand factual infomiation 






Drew cer«yusJoit», 
maitf infe^ncss, 
deduce tn^nirtgeeoi 



infer traits, feelings or 
motivations of characters 



Apply Info from text to new 
situation, depict Info In new 
form, or predict outcome 



Oetermine-main idea or topic 






Interpret non<lilcral language 



100 



90- 



80- 



70- 



60- 



S 50- 



40- 



30- 



20 - 



10 - 



o-h- 



Percent of Questions 
on fTBS 



Read amounts, 
compare quantities, and 
interpret relationships 



'andfdeBn^^eq6kli^ 
and inequalities 



Add whole numbors, 
subtract whole numbers, 
multiply whole numbers, 
divide whole numbers 



Single step: add and subtract 
single step; multiply and 
divide, multiple step, and 
problem-solving strategics 



PercentTime Listed 
as instructional Focus 



Read amounts, 
compare quantities, and 
interpret relationships 



5 ' Nuiperptibrn^arfthmeU^ 




Addwhole numbers, subtract 
whole numbers, multiply 
whoienumbersidivide 
v^afOhumbeis 



Single step: add 
and subtract, 
single step' multiply 
and divide, 
multiple step, and 
problem-solving 
strategies 



Analysis was conducted on the 1999 Summer Bridge curriculum. For reading, we report the breakdown of questions on the ITBS as provided in Uie Summer Bridge 
Curriculum. For math, we referred to the booklet titled "Iowa Tests of Basic Skills: Content Classifications with Item Norms." Percent time listed as instructional focus 
was obtained from a count of the proportion of days in which the focus is listed. When there were multiple foci, a weighted average was calculated. 
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Instructional Foci from Summer Bridge Curriculum Instructional Foci from Summer Bridge Curriculum 

Sixth-Grade Mathematics Eighth-Grade Mathematics 



100 n 



90 “ 



80 - 



70 - 



60 



50 - 



40 - 



30 - 



20 - 



10 



0 -h- 



Percent of Questions 
on ITBS 

Road amounts 
compare quantities 
Interpret relationships 
and trends 



Percent Time Listed 
as instructional Focus 



Percent of Questions 
on tTBS 




Whole mtmbsf s, 
{f act Ions, 

dfecirnais 



Singio-stop 

multiplo-stop 

problem-solving 

strategies 



Read amounts 
compare quantities 
interpret relatlonshipe 
and trends 



.aeomatty;: TMf -j 






WhoNt humbero. 
froctlohs* 
dactm^s 



Single-step 

muitiplo-stop 

problem-solving 

strategies 



100 - 
90 “ 
80 - 
70 - 
60 - 
I 50 - 

Ol 

40 - 
30 - 
20 - 
10 - 



■ 0 



Percent Time Listed 
as Instructional Focus 



liiead amounts 
compare quantities 
interpret relationships 
and trends 




SlnglO'Stcp 

multiple-step 

problem-solving 

strategics 



Single-step 

multiple-step 

probicm-sotvlng 

strategies 



Anaiysis was conducted on the 1999 Summ^ Bridge curriculum. For reading, we report the breakdown of questions on the ITBS as provided in the Summer Bridge 
Curriculum. For math, we referred to the booklet tilled "Iowa Tests of Basic Skills: Content Classifications with Item Norms." Percent time listed as inslrucUonal focus 
was obtained from a count of the proportion of days In which the focus is listed. When there were multiple foci, a weighted average was calculated. 
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Appendix 6 

The Summer Bridge mathematics curriculum was initially organized like the reading curriculum, drawing on a 
variety of workbooks and texts. The curriculum was rewritten in 2000 for the sixth and eighth grades (third 
remained the same) so that fewer workbooks were used and the curriculum was more self-contained. Most 
teacher-directed activities and student lessons were provided in one set of materials, and teachers had more 
flexibility in pacing the curriculum. We provide examples of typical lesson plans from the Summer Bridge third, 
sixth-, and eighth-grade mathematics curriculum. 



Sample Daily Lesson Plan from the Summer Bridge Curriculum, 1999 Lesson: 19 

Third-Grade Mathematics 



instructional Focus 



Data Interpretation - Bar Graphs 



Teacher-Directed Activity 



Just the facts!! Write the folfowing words on the chalkboard; 

data key ptctograph table tally mark 

Read the following and have students raise their hands to fill in the blanks: 

1 . Information about people or things that can be organized and sorted is called (data) 

2. A graph that shows data by using pictures that stand for more than one thing is a . (pictograph) 

3. You can organize data in a , (table) 

4. A at the bottom of a pictograph tells how many each picture stands for. (key) 

5. A is used to count data, (tally mark) 



Daily Challenge 



Ask: Whicti fbod do you like best— hof dogs, hamburgers, tacos, or pizza ? 

Have students raise their hands to show their vote. Record on the chalkboard each food item and the count. 

Tell the students we will use a bar graph to record our data today. Have the students look up bar graph in their “Math Tools." 
Ask a volunteer to read the definition. Explain that a bar graph has a scale of numbers to help read what each bar shows. The 
bars can be horizontal or vertical. Horizontal bars go across from left to right. In a vertical bar graph, the bars go up. 

Write the following title on the chalkboard: Our Favorite Foods 

Ask: What will our labels be? (Answer: Hot dogs, hamburgers, tacos, pizzas.) 

Use 2s for the number scale. Develop a bar graph on the chalkboard using the data already listed on the chalkboard. 

Ask the following questions: 

1 . How many students picked hot dogs as their favorite food? (A) 

2 Which food is liked the best by the most students? {pizza} 

3. What scale is used on the bar graph? (numbers by 2s) 

4. What type of bar graph is this? {bor\zonia\) 

Have the students open their Take Another Look io pg. 92-93, Read and discuss the top of the pages with the class. Work 
through the problems with the class. 



Taking A Break 



Divide the class into groups of 4. Have half the class develop a horizontal bar graph and the other half of the class develop a 
vertical bar graph describing the class's "Favoiite Sports/’ 



Test Prep 



Have students open Get Ahead in Math to p.47. Read and discuss the Mini-Lesson with the class. The students should use 
“Math Tools" to review the vocabulary terms. Work through each problem in Practice. Allow students time to solve the 
problems. Discuss the solutions as a class. Assign Test Yourself. Have students use the answer box for the answers. 



Source: Chicago Public Schools 1 999 Summer Bridge Teacher Handbook, Grade 3. Lessons have been condensed. 
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Sample Daily Lesson Plan from the 
Summer Bridge Curriculum, 2000 

Sixth-Grade Mathematics 



Instructional Focus 



Data Interpretation - Bar Graphs 

• Read and interpret bar graphs to solve problems. 

• Compare information on bar graphs. 

• Analyze patterns and trends, and predict changes 
based on data found in bar graphs. 



Concepts: Reading Bar Graphs 



Point out bar graphs provide a good visual picture 
of data. 



Suggestions 



Before you work through the examples on this page, 
be sure that students are reading the bar graphs 
correctly. 

If students are having difficulty reading across to the 
number on the vertical axis, encourage them to lay a 
ruler or other straight edge across the top of the bar. 

Example: Students must identify the Information to be 
compared on the graph. The example asks them to 
compare people in 1998; therefore, they should be 
looking only at the three bars for 1998. 



Skills: Interpreting Bar Graphs 



Explain to students that some bar graphs can show 
trends in the data. For example, are the data generally 
rising or falling? So, a graph can sometimes be used to 
predict the future. 

Example: Point out that the questions they will be doing 
call for a prediction of the future, so students will have to 
find a trend in the data and continue that trend into the 
year 2000. Remind students to focus on which particular 
bars they should be examining or comparing. 



Problem Solving: Interpreting Bar Graphs 



Point out that the vertical axis is in thousands. There- 
fore, the line labeled 20 represents not 20, but 20,000. 

Ask students how they would estimate the height of a 
bar that ends between two lines. 



Skills Review 



The problems on this page review skills introduced In 
this lesson and in earlier lessons and prompt students 
to: 

• Read and interpret bar graphs. 

• Compare data. 

• Use inverse operations to find missing numbers 
in an equation. 



Source: Chicago Public Schools 2000 Summer Bridge Teacher Handook, 
Grade 6. Lessons have been condensed 



Sample Dally Lesson Plan from the 
Summer Bridge Curriculum, 2000 

Eighth-Grade Mathematics 



Instructional Focus 



Data Interpretation — 100 Percent Bar Graphs 

• Interpret 100 percent bar graphs and their keys. 

• Estimate the size of each section on a 100 percent 
bar graph. 

• Compare the sizes of different sections for one time 
period or the same section for different time periods. 

• Predict trends based on a comparison of data on 100 
percent bar graph. 



Concepts: Understanding a 100 percent Bar Graphs 



Point out that on 100 percent bar graphs, since the 
length of each bar equals 100 percent, you compare 
amounts not by the length of the whole bar but by the 
length of each section that makes up the bar. You may 
want to draw the following graph (a typical bar graph) on 
the chalkboard to help students see how a 100 percent 
bar graph could be shown as separate bars. 



Suggestions 



Before students try to answer the questions shown in 
the examples, make certain that they can estimate the 
sections of each bar by asking questions such as these: 

• In 1970, what percent of the school budget came 
from miscellaneous sources? 

• In the same year, what percent came from the PTA? 

• The PTA bar stops at about the 10 percent line. Why 
isn’t the PTA percent 10 percent? 

Explain that students can find each percent in the same 
way: by estimating where the line for that section begins 
and ends and subtracting the two numbers. 



Skills: Reading a 100 Percent Bar Graph 



Because this bar graph shows intervals of 20 percent, 
students may have trouble estimating the value of each 
section of each bar. For example, to find the percent 
spent for housing in 1985, students must first determine 
that the section begins between 60 percent and 80 
percent. Next, they must realize that it begins closer to 
80 percent than to 60 percent, so a good estimate is 
75 percent. 



Skills Review 



The problems on this page review skills introduced in 
this lesson and prompt students to: 

* Interpret 100 percent bar graphs and their keys. 

* Estimate the size of each section on a bar graph. 

* Compare the size of different sections for one time 
period or the same section for different time periods. 

* Predict trends based on a comparison of data on a 
100 percent bar graph. 

* Solve word problems based on information in 100 
percent bar graphs. 

Source: Chicago Public Schools 2(K)0 Summer Bridge Teacher Handbook, 
Grade 8. Lessons have been condensed. 
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APPENDIX C 

Methodology for Estimating Summer Bridge 
Test-Score Increases 

Adjusted Summer Bridge learning gains are obtained by subtracting students' observed August test score from 
a predicted May test score. We obtain this predicted gain by estimating a student growth model through a 
hierarchical linear model (HLM) that uses data on a student s entire testing history and on the learning trends 
of students in both pre- (1992' 1996) and post- (1997-1999) policy cohorts. A model is estimated separately for 
each grade (third, sixth, and eighth) and for each subject (reading and mathematics). I’he outcome variable is 
the students test score in each year in that subject on the Iowa Tests of Basic Skills measured in the grade 
equivalent (GE) metric. For the purposes of illustration, we describe the sixth-grade model below. 

Level 1 is a repeated measures model in which Y..^ is the achievement at grade i, for student) in school k (the 
school attended in grade 6) as shown below: 

(1) (Grade) + (Sixth) + (Repeat) + e.., 



The grade variable is centered so that it takes on the value of zero in the fifth grade. Sixth is a dummy variable 
that equals one in the promotional gate grade and zero otherwise. The coefficient of this variable measures the 
extent to which the students sixth-grade test score deviated from the score that would have been expected based 
on the students initial status and learning trajectory up to that point. Repeat is a dummy variable that equals 
one when a pre-gate grade is repeated. This controls for any prior retention experience in estimating each 
individual s growth trajectory. 

Level 2 models the coefficients from the individual growth trajectories as a fiinction of student characTreristics. 

, , J^ojk = Pook + Poik (Year94) + • • • + (Year99) + r ^ 

( 2 ) 

’^ijk =Piok+Piik(Year94) + • • • + P,g^(Year99) + 

= P 20 k + P 2 ik (Year94) + • • • + P^gk (Year99) + 

Jtsjk = P 3 Qk (assumed fixed for simplicity) 

Because the random effects — and — allow the initial status, learning rate, and sixth-grade deviation 

to vary across students, we have a separate learning curve for each student. The year variables are dummy 
variables for each cohort in the sample. Year93 is omitted so the coefficients on the other year indicators repre- 
sent the extent to which initial status and linear learning rate varied from 1993. Thus, this model allows us to 
estimate growth curves for each student across time, taking into account that cohort ability levels and achieve- 
ment growth within grades may be changing over time because of more general changes in the school system 
(i.e., the effect of other reforms or general improvement in the school system in test scores, differences in cohort 
characteristics such as more affluent students entering the public school system, or the expansion of kindergar- 
ten or pre-kindergarten programs). In this model, the third or scliool level is left unconditional and simply 
serves to correctly estimate the standard errors. 
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Using the estimates from this model, we can calculate each student’s predicted sixth-grade May test score as 
Y,j, = (;ro., + r’ojk) + (;r,j, + r,;) 

where JtQjj^znd the predicted fifth-grade score and average annual learning gain respectively, based on 

the model above for students in a particular cohort and school, and r*ojk and are estimated Bayes residuals 
for each student. For students who participated in Summer Bridge, the adjusted or ''true” summer gain 
in the sixth grade is then calculated by subtracting their observed August test score ( y^hridgejk ) fro™ their 
predicted May test score {Y(,jj^)^ We call this the adjusted summer gain, 

(4) AY^.,^ = ~ ^6jk 

We use a slight variation of the model above to gain an estimated school-year gain for students who attended 
summer school. Rather than estimating cohort effects, in this second model we estimate a differential effect for 
students who attended summer school in each yean Thus, at the student level (Level 2) we replace equation 2 
with: 

"ojk =Pook +Poik(Summer97)jk+Po2k(Summer98).|^ +Po3i^(Summer99)ik + 

(5) ",jk = Piok + P„k(Summer97)jk+Pi2k(Summer98)j|^ + P,3,^(Summer99)jk + V 
^2jk =P20k +P2ik(Summer97)jk+P22k(Summer98)j|^ +P23k(Summer99)ik + r^jk 
^ 3 jk “ Psok (^sumed fixed for simplicity) 

The average school year gain prior to the promotional gate grade for students who attended summer school in 

1997 is then P,o;t+Piu 
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APPENDIX D 

Methodology for Estimating Effects of 
Student and School Characteristics on Student Test-Score Gains 



The statistical analysis reported in Chapter 2 is based on a two-level hierarchical linear model (HLM) of the 
adjusted Summer Bridge test-score gains. We used the same model for grades three, six, and eight for reading 
and mathematics. 

At Level 1, the independent variables were dummies for being African-American, Latino, or male, an SES 
(socio-economic status) variable based on census information, age, and being at risk of retention based on prior 
achievement. High risk was defined as having a latent pre-test score a half year to one and a half years below the 
cutoff score and very high risk was having a latent pre-test score more than a year and a half below the cutoff score. 

At Level 2, we used school- and grade-level predictors. Because we were unable to link students to their exact 
Summer Bridge teacher and classroom, we aggregated classroom information to the grade within the school. In 
many schools, the grade and classroom were the same, as the school had only one Summer Bridge classroom for 
that grade. For the teacher measures — evaluation of Summer Bridge curriculum and individualization of in- 
struction — we took the mean of the teacher responses for that grade if there was more than one teacher per 
grade. The analysis used race and achievement levels of the school, programmatic characteristics (such as class 
size, having separate teachers for mathematics and reading, and the percentage of students that teachers knew 
prior to Summer Bridge), grade characteristics (such as passing and exclusion rates) and teacher characteristics 
(such as having a Master's degree or higher, having taught Summer Bridge before, years of teaching experience, 
and subject taught during the school year). 

Level 1 (students): 

Y..= (3y. (SES) + P,. (African-American) + P^. (Latino) + P^.(Male) + P^. (Attending Home 
School) + (Age) + P^. (High Risk) +Pg. (Very High Risk) + e.. 



where T.is the adjusted Summer Bridge gain. 



Level 2 (school level): - 

p^^. = (Evaluation of Bridge Curriculum) + Y(p (Individualization of Instruction) + (Teacher 

w/ Masters Degree) + (% Students Teacher Knew) + y ^5 (Separate Subject) + (Years Teaching) + 

Y^^ (Science/Social Studies Teacher) + y^^^ (Non-teacher) + y^.^ (Taught Bridge Before) + (% 

Excluded in Classroom) + y^^^^ (Passing Rates) + y^^^ (Very Low-Achieving School) + y^^^^ (Low- 
Achieving School) + y^j^^ (Small Class) + y^^^^ (Large Class+) (Predominantly African-American) + 
y^^j„ (Predominantly Latino) + y^j^g (Mixed Race) + (Predominantly Minority) + r^^^. 

R = y 6 = V 

I 1) MO L^8} ‘80 




BEST COPY AVAILABLE 

125 



RESULTS FROM SUMMER BRIDGE 125 

APPENDIX E 

Methodology for Developing Survey Measures 

The Consortium uses Rasch analysis to develop measures from questions on its biannual surveys. Each measure 
surrounds a single concept such as personalism or academic press. The Rasch model is a member of the family 
of item response or latent trait models. (The model is based on measuring a single trait, following the concept 
that the trait can be examined using a unidimensional scale defined by an ordering of items.) Each survey item 
is assigned a position on the scale based on how frequently or how likely the question is endorsed. In addition, 
all persons responding to the survey items are assigned a position on the same scale based on their responses to 
those questions. This allows all respondents to the survey to be given a measure that indicates the extent to 
which they display the trait or attitude being examined. The person measures and the item difficulties are given 
in the logit metric (log odds units); but for reporting purposes, the logits are converted to a 0 to 10 scale. 

Measure development begins with selecting questions from the surveys that seem conceptually coherent. Con- 
sortium analysts then use the Rasch model to analyze the survey items and those that fit statistically and are 
conceptually coherent are kept in the measure. The Consortium developed the measures on academic press and 
personalism using jts 1994 student surveys. The items from these measures were repeated in subsequent surveys 
in 1997 and 1999, and corresponding measures were made. The item difficulties produced in the 1994 Rasch 
analysis were used to produce measures for the other years, making the measures directly comparable across 
years and useful for longitudinal analysis. Similarly, in developing the Summer Bridge measures of academic 
press and personalism, the 1994 item difficulties were used so that the spring 1999 and Summer Bridge 1999 
measures could be compared. In this case, the Consortium was able to measure attitudes for each student 
during the school year as well as for during Summer Bridge. 
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APPENDIX F 

Methodology for Estimating Student Reports on Summer Bridge 

The statistical analysis reported in Chapter 3 is based on the two-level hierarchical linear model (HLM) used 
previously in the analysis of Summer Bridge test-score gains. We used the same model for each of the three 
outcome variables: the difference between students’ reports on personalism between Summer Bridge and spring 
1999 , the difference in tlieir reports of academic press, and students’ evaluations of Summer Bridge. The analy- 
sis was run separately for sixth and eighth grades. 

At Level I, the dependent variables were dummies for being African-American, Latino, or male, an SES (socio- 
economic status ) variable based on census information, age, and being at risk of retention based on prior 
achievement. High risk was defined as having a latent pre-test score a half year to one and a half years below the 
cutoff score and very high risk was having a latent pre-test score more than a year and a half below the cutoff score. 

At Level 2 , we used school- and grade-level predictors. Because we are unable to link students to their exact 
Summer Bridge teacher and classroom, we aggregated classroom information to the grade within the school. In 
many schools, the grade and classroom are the same, as the school had only one Summer Bridge classroom for 
that grade. The analysis used race and achievement levels of the school; programmatic characteristics, such as 
class size, having separate teachers for mathematics and reading, and the percentage of students that teachers 
knew prior to Summer Bridge; grade characteristics, such as passing and exclusion rates; and teacher character- 
istics, such as having a Masters degree or higher, having taught Summer Bridge before, years of teaching expe- 
rience, and subject taught during the school year. 

Level 1 (students): 

Y..= Py. + Pj. (SES) + P^. (African-American) + P^. (Latino) + P_j.(Male) + P^. (Attending Elome School) + 
p;. (A^e) + p,. (High Risk) + p^. (Very High Risk) + e. 



Level 2 (scliool level): 

(Teacher w/ Master s Degree) + (% Students Teacher Knew) + y„ 3 (Separate Subject) + 

Y^j 4 (Years Teaching) +y^j^ (Science/Social Studies Teacher) + Y^^; (Non-Teacher) + Y^^ (Taught Bridge Before) 
+ Y(,jj (% Excluded in Classroom) + Yj,p (Passing Rates) + You) (Very Low-Achieving School) + (Low- 
Achieving School) + Yoj 2 (Small Class) + y^| 3 (Large Class) + y^j 4 (Predominantly African-American) + 
Yjjj 5 ( Predominantly Latino) + Yo,^ (Mixed Race) + Y(,}7 (Predominantly Minority) + r^^^^ 

Pi) ~ Yio. .... Psj ” Ygo 
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APPENDIX G 







Sixth Grade 






Eighth Grade 






Tested 


Summer 

surveys 


Matched 

sample 


Tested 


Summer 

surveys 


Matched 

sample 


Number of students 


7,265 


4,829 


2,519 


6,128 


4,225 


2,303 


Number of schools 


321 


289 


240 


280 


247 


207 


Student Characteristics 














African-American 


64% 


61% 


53% 


61% 


57% 


50% 


Latino 


31% 


33% 


41% 


34% 


37% 


44% 


Average Age 


13 


13 


13 


15 


15 


15 


Recent bilingual program 


5% 


8% 


8% 


5% 


7% 


7% 


Average Spring ITBS reading 


4.8 


4.8 


4.8 


6.8 


6.8 


6.9 


Average Bridge ITBS reading 


5.2 


5.3 


5.3 


7.5 


7.5 


7.5 


School Characteristics 
Predominantly 
African-American 


56% 


56% 


52% 


56% 


54% 


50% 


Predominantly Latino 


14% 


13% 


15% 


13% 


14% 


16% 


Predominantly minority 


12% 


12% 


13% 


13% 


13% 


15% 


Mixed Ethnicity 


8% 


8% 


9% 


8% 


8% 


9% 


At national norms 














reading (1999) 


29% 


30% 


30% 


29% 


30% 


31% 
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APPENDIX H 

Statistical Analysis of Teacher Summer Bridge Measures 

The statistical analysis reported in Chapter 4 uses a two-level hierarchical linear model (HLM) with teachers at 
Level 1 and schools at Level 2. We used the same m odel for each of the three outcome variables: teacher reports 
on the quality of the Summer Bridge curriculum, extent of individualized instruction, and reports of student 
motivation. 

At Level 1, the independent variables were classroom characteristics such as dummies for having tutors, grade 
level, and class size; a variable for passing rates; and characteristics of the teacher, such as having a masters 
degree or higher, having taught Summer Bridge before, years teaching experience, and subject taught during the 
school year. 

At Level 2, we used the following as predictors: school-level racial composition, SES, percent excluded, having 
separate teachers for reading and mathematics, and being a low- or very low-performing school. 

Level 1 (teachers): 

Y..= +Pj. (Tutor) + (% Students Teachers Knew) + P^. (Masters Degree +) + P^ (Taught Bridge 

Before) + P^. (Years Teaching) + P^. (Science or Social Studies Veacher) + P„. (Non-Core Subject Teacher) 

+ pg. (Grade 3 ) + P,jj (Grade 8) + p^^j. (Average Gain Needed to Pass) +pjj. (Small Class)+ p^^j (Large Class)+ 

Pj3- (Passing Rate)+ 

Level 2 (school level): 

Poj “ Yoo Yol (Separate Subject) + Yq 2 (School SES) + 7^3 (% Excluded) + 7^4 (Very Low-Achieving) + 

7^3 (Low-Achieving) + 7^^^ (Predominantly African-American School) + 7^^ (Predominantly Latino School) 

+ 7^g (Mixed Race School) + 7^,^ (Predominantly Minority School) + r^^ 

Pl|" Y|o Pl3j “ Yi3o 
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Methodology for Coding Classroom Observations 

Classroom observations were coded to determine the extent to which teachers followed and used the daily 
lesson plans; provided instruction that was clear, accessible and accurate; engaged students and encouraged 
participation; and provided individualized and targeted instruction to meet their students’ needs, Reading and 
mariiematics lessons were coded separately for each observation, as the subjects were sometimes taught by 
different teachers, and we assumed that teachers might have more expertise in one subject than the other.. Each 
classroom observation was coded independently by two raters for a total of three raters participating in the 
analysis. The majority of the time the independent raters agreed on observation codes. When there was dis- 
agreement between the two initial raters, a third rater was asked to code the observation independently. The 
final code in these situations was the code two of- the three raters selected. A third rater was needed for 11 
percent of mathematics lessons and 13 percent of reading lessons. In all cases where a third rater was needed, the 
rater agreed with one of the two initial codes. 

In coding the observations, we initially grouped lessons into six categories, ranging from classrooms where 
virtually no instruction took place (insufficient instruction) to classrooms where teachers not only implemented 
the daily lesson correcdy, but also provided instruction targeted to the needs of their students, taught concepts 
in multiple ways, and ensured that the vast majority of students understood the lesson (tailored instruction). 
After coding was completed, the six groups were reduced to four. Initially, we made distinctions between class- 
rooms where teachers provided no instruction whatsoever and had highly chaotic classrooms, provided little to 
no instruction but maintained a whole class structure, and provided little to no instruction but had students 
work individually for most of the class. We found that these distinctions did not necessarily hold and that 
classrooms where teachers provided little or no instruction were homogenous enough to be grouped into a 
single category. 
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APPENDIX J 

Methodology for Estimating Test-Score Gains 
for Summer Bridge Observation Schools 

The statistical analysis reported in Chapter 5 is based on a two-level hierarchical linear model (HLM) of the 
adjusted Summer Bridge test-score gains used in Chapter 2 . We used the same model for grades three, six, and 
eight for reading and mathematics. The analysis was run on the entire sample of Summer Bridge schools, and 
we used the empirical Bayes estimates of the school-level gains to evaluate the observation schools. 

At Level 1 , the dependent variables were dummies for being African-American, Latino, or male, an SES (socio- 
economic status) variable based on census information, age, and being at risk of retention based on prior 
achievement. High risk was defined as having a latent pre-test score a half year to one and a half years below the 
cutoff score and very high risk was having a latent pre-test score more than a year and a half below the cutoff score. 

At Level 2, we used school- and grade-level predictors. Because we are unable to link students to their exact 
Summer Bridge teacher and classroom, we aggregated classroom information to the grade within the school. In 
many schools, the grade and classroom are the same, as the school had only one Summer Bridge classroom for 
that grade. The analysis used school race and achievement levels, and grade characteristics such as passing and 
exclusion rates. Unlike the model in Chapter 2 , we do not have measures of programmatic characteristics such 
as class size and the percentage of students that teachers knew prior to Summer Bridge or teacher characteristics 
because the Summer Bridge survey was administered only in 1999, and the classroom observation was con- 
ducted in 2000 . 

Level 1 (students): 

Y..= -I- pj. (SES) + p 2 - (African-American) + p^. (Latino) + P^.(Male) + p^. (Attending Home School) + 

P(!j (Age) + I?,. (High Risk) + (Very High Risk") + e,. 

where K.is the adjusted Summer Bridge gain. 

Level 2 (school level): 

Po.= Yoo '*■ Yoi Excluded in Classroom) + y^^{Vcry Low-Achieving School) + 7^3 (Low-Achieving School) 

+ 

(Passing Rates) +Y 05 (Predominantly African-American) + (Predominandy Latino) + Yq? (Mixed 
Race) + Y(^g (Predominantly Minority) -t- r^^ 

Pij ” Yio p8j “ Yg() 
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Four-Year Test-Score Trends for Summer Bridge High Scorers and 
Comparison Groups, Pre- and Post-Policy 
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Intfodudioii and Report Overview 

' Johnston (2000); Kasindorf and Howletr (2001); Mathews (2000). 

^ Gewertz (2000). 

^ Kasindorf and Hewlett (2001). 

^ Denham and Lieberman (1980); Levin andTsang (1987); Smith (1998). 

' Alexander, Entwisile, and Olsen (200 1 ); Cooper et al. (1996); Enrwisile and Alexander 
(1992); Heyns (1978); and Heyns (1987). 

Cooper et al. (2000). 

^ Cooper et al. (2000). 

^ House (1998); Roderick et al. (1999). 

Monitors are external — meaning sent from the central office — rather than princi- 
pals or fellow teachers. 

Roderick et al. (1999). 

" Roderick et al. (2000). 

Chapter 1 

‘ CPS elementary schools usually include kindergarten through eighth grade. 

^ Students who are in special! or bilingual education may attend Summer Bridge, but 
their teacher would make this decision. For students with disabilities, teachers follow 
the students Individual Education Plan. The decision not to include students who 
were in bilingual or special education meant that many students were nor included 
under the policy. In 1997, only 70 percent of third graders and 80 percent of sixth 
and eighth graders were included. Inclusion rates have decreased over time, 

^ For the Consortiums first report on ending social promotion, a statistical model 
was developed to estimate each students risk of not meeting the cutoffs based on his 
or her entire test-score history. If a srudents test score wiis abnormally low in the pre- 
promotional gate years (second, fifth, and seventh grades), this method would cor- 
rect for that, as it predicts the students test score based on his or her ITBS score in 
every prior grade, thus providing a more accurate prediction of a students underly- 
ing ability. 

^ Easton et al. (1 999). 

^ Roderick et al. (1999). 

^ Exceptions were granted for schools that were undergoing construction or schools 
with very few students required to attend. 

^ The official average CPS elementary school class size is 23 its reported on the CPS 
website. Fhis figure, however, includes special education classrooms, i’he typical class 
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size is probably closer to the 29 to 3 1 number stated in the 
union contract. 

® See Appendix A for the instructional foci of sixth-grade read- 
ing as well as mathematics for all three grades. 

^ See Appendix B for a brief description of the mathematics 
curriculum and for examples of daily mathematics lesson plans. 

Summer Bridge Teacher Handbook (1999), Reading, 
Grade 8. 

Chapter 2 

‘ The analysis presented in this chapter was developed by Me- 
lissa Roderick, Brian Jacob, and Anthony S. Bryk, and was 
originally presented in a paper that will be published in G.D. 
Borman and M. Boulay, Eds. (forthcoming). 

^ We excluded students who were retained and are participat- 
ing in Summer Bridge for a second time because we wanted to 
examine the effect of Summer Bridge independent of the ef- 
fect of retention. We included only those students who needed 
to pass cutoff scores on a particular test. Roderick, Jacob, and 
Bryk (forthcoming) found that test-score increases among sLxth 
and eighth graders who did not need to pass the cutoff in that 
subject actually declined over the summer. This effect was larger 
in reading than in mathematics and did not occur among third 
graders. These declines in test scores for students who had al- 
ready met the standard in that subject before attending the 
program suggest that students who did not need to pass the 
exam in order to be promoted may have exerted less effort on 
that exam. Because it is difficult to disentangle test motivation 
from learning among this group, we excluded them from the 
analysis of test-score gains. 

^ In 2000, the cutoff scores in sixth grade increased from 5.3 
to 5.5 and in eighth grade from 1 A to 7.7. 

Cooper et al. (2000). 

^The standard deviation measures the spread of students’ test- 
score gains around the mean, calculated by taking the square 
root of the squared differences between the test-score gains 
and the mean gain, divided by the number of students. 

Cohen (1969) describes an effect size of 0.2 as small, 0.5 as 
medium, and 0.8 as large. Still, interpreting the effectiveness 
of a particular intervention can only be meaningful in relation 
to other similar interventions, A 'small” effect size of 0,2 may 
in fact produce a very noticeable difference. 

" This is calculated by dividing the adjusted gain by the stan- 
dard deviation of test scores in the subject in August. Cooper 
et al. (2000) uses an effect size calculation that is the average of 
the pre- and post-test standiud deviation. Because our adjusted 
gain is estimated on the basis of a predicted test score that 
corrects for year-to-year fluctuations in student performance, 
the standard deviations of the predicted test scores are sub- 
stantially below what would be expected from one test admin- 
istration to the next. Therefore, to be conservative, we use the 
post-test standard deviation only, 

^ Smith (1998). 

Easton et al. (2000). 
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Schools: where over 85 percent of the student body is Afri- 
can-American .were considered to be predominantly African- 
American for this analysis. Similarly, schools where over 85 
percent of students are Latino were categorized iis predomi- 
nantly Latino. 

Chapter 3 

^ Cooper et al. (2000), Finn md Achilles (1999). 

^ For example, in a 1999 Consortium report entitled, Social 
Support, Academic Press, and Student Achievement: A View from 
the Middle Grades in Chicago. Valerie Lee and her colleagues 
found that both the level of academic standards for perfor- 
mance (academic press) In classrooms and the extent to which 
students felt that teachers were personally attentive to their 
learning needs (personalism) were strongly related to student 
achievement. This is one of several studies that have empha- 
sized the importance of combining a press for student achieve- 
ment with personal support. See Bryk, Lee, and Holland 
(1993); Noddings (1988); Phillips (1997); Shouse (1996). 

I he Summer Bridge measures for academic press and per- 
sonalism use the same set of items as measures made from the 
1 999 student survey given to sixth- and eighth -grade CPS stu- 
dents in the spring. To make the Summer Bridge and school- 
year measures comparable, we anchored the Summer Bridge 
measures on the item and step difficulties from the school- 
year measures. Because the school-year measure had separate 
questions for math and reading, we used the mean of the step 
and item difficulties. Differences in item difficulties for the 
two subjects were minimal. 

^ Assessment of Summer Bridge is a Rasch measure that in- 
cludes items from the Summer Bridge survey comparing Sum- 
mer Bridge to the regular school year in terms of liking school, 
working harder, learning more, teachers having more time, stu- 
dents helping more, and reporting whether reading was too 
hard, math was too hard, summer school was boring, summer 
school was fun, if students were mad that they had to go to 
summer school, and if they hated summer school. 

^ Boys had a statistically significant difference in their assess- 
ment of Summer Bridge and in personalism but not in aca- 
demic press. In their assessment of Summer Bridge, sixth -grade 
boys are 0. 1 4 lower than girls and eighth-grade boys are 0. 18 
lower. For the personalism measure, sixth-grade boys are 0,25 
lower and eighth-grade boys are 0,23 lower than girls. 

Maclver, Reuman, cmd Miiin (1995). 

^ Ames (1992); Harackiewicz, Abrahams, and Wageman 
(1987); Maclver, Stipek, and Daniels (1991); Stipek (1996). 

Chapter 4 

' Half of teachers in our 2000 summer interviews who reported 
a substantially smaller class compared to the school year also 
noted that the small class size allowed them to provide indi- 
vidualized attention to students. Approximately 27 percent 
noted that they were able to make more personal connections 
with their students and that reduced class size allowed them to 
better group students for instruction. 
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“ Class size was measured by whether students were in a very 
small Summer Bridge class (12 or fewer students), an average 
class (13 to 17 students), or a large class (1 8 or more students). 
The achievement level of the class- or grade was measured by 
how far behind the reading test-score cutoff the average stu- 
dent was on entry into Summer Bridge. Our analysis also took 
into account additional teacher and school characteristics in- 
cluding teachers’ years of experience, whether the teacher had 
taught Summer Bridge before, whether the teacher had a 
miister s degree or higher, was a science or social studies versus 
a regular elementar\' school teacher, or was not a classroom 
teacher. Other school-level characteristics included the aver- 
age socio-economic status of the school and the percentage of 
students in the promotional gate grade excluded from the test- 
ing polic)". 

Chapter 5 

‘ We conducted this analysis for reading only. In 2000, the 
sixth- and eighth-grade mathematics curriculum was re- 
vised to include more than one lesson per day, and it be- 
came difficult to determine what lesson teachers were 
supposed to be on. 

^This scheme was used in BetsAnn Smiths 1998 Consortium 
report, h's About Time. 

^ Instructional set-up time would include activities like pro- 
viding iin example while preparing students for an exercise; 
non-instructional set-up time includes handing out materials, 
having students take out books. 

This analysis used a two-level hierarchical linear model. At 
the student level it controlled for socioeconomic status, age, 
race, gender, whether students attended their home school for 
Summer Bridge, and the latent prior year ITBS score. At the 
school level the model controlled for school achievement, pass- 
ing rare, and racial composition (see Appendix J for details 
and model). 

Chapter 6 

‘ Cooper et ai. (2000). 

“ This chapter draws on the work of Brian Jacob cmd Lcirs 
Lefgren looking ar the sustainability of Summer Bridge (2002). 
We are indebted to their intellectual contribution to this work. 



^The band was determined using the distribution of test scores 
and possible grade equivalent scores for each exam. 

We excluded 664 third graders and 530 sixth graders in 1997, 
and 923 third graders and 5 10 sixth graders from our Summer 
Bridge high scorer group because they were retained the fol- 
lowing school year. 

^ There are several well-known shortcomings of using GE scores 
for studying growth over time. Different forms of the exam 
are administered each year. Because the forms are not correctly 
equated, one might confound changes in test performance with 
changes in form difficulty. The GE is also not useful for assess- 
ing student growth across grades because scores are not di- 
rectly comparable across test levels. Since the GE is not a linear 
metric, a score of 5 3 on level 12 (grade 6) of the exam does 
not represent the same thing as a score of 5.3 at level 14 (grade 
8). Finall}', the GE metric is not linear within test le\'^els be- 
cause the scale spreads out more at the extremes of the score 
distribution. One additional correct response ar the top or 
bottom of the scale can translate into nearly a gain of one full 
GE, whereas an additional correct answer in the middle of the 
scale would result in only a fraction of this increase. Thus, if 
Summer Bridge students, who are low performing, move to 
the middle of the .scale, it would appear that their learning 
gains decrease. over time. 

Bryketcil. (1998). 

^ In pre\dous reports, we used the 1995 cohort as a compari- 
son group when looking at one-year post-gate trends. How- 
ever, because we are looking at four-year trends in student 
performance, sixth graders in the 1995 cohort were included 
in the policy for two years (seventh and eighth grades). 
Roderick, Jacob, and Bryk’s (forthcoming) analysis of pre-policy 
effects found greater than average gains in both the gate (sixth 
and eighth grades) and pre-gate (fifth and seventh grades) years. 
In order to avoid confounding effects caused by the imple- 
mentation of the polic)^ with general trends in learning, we 
did not use the 199,5 cohort as a comparison. Sixth graders in 
1994 did, however, face the policy in eighth grade in 1996, 
the first year of implementation. 

^ Based on 1998 learning gains for sixth graders. 

Interpretive Summary 

^ Roderick, Jacob, and Bryk (forthcoming). 
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COMMENTARIES 



Geoffrey D. Borman 

Ufiiyersitf of iisconsin-l/ladison 



Over its recent history, the Chicago Public Schools (CPS) has often served 
as a national exemplar of educational experimentation and innovation. 
Beginning with the grass- roots decentralization reform of 1988, then the 
major reorganization of the central office in 1995, and, most recendy, the 
introduction of high-stakes accountability along with the Summer Bridge 
program, elementary and secondary education in Chicago has attracted 
considerable national attention. Since 1990, the Consortium on Chicago 
School Research has been a vital component of these reform efforts. Work- 
ing collaboratively with the school system, the Consortium has brought 
high-quality evidence to bear on matters of educational policy and prac- 
tice and has helped fuel reform by stimulating a democratic discourse about 
“what works” in reforming Chicago public schools. This model of educa- 
tional experimentation informed by balanced and high-quality research is, 
once again, demonstrated in the report Ending Social Promotion: Results 
from Summer Bridge. 

The phenomenon is one that has swept the country: school systems are 
putting an end to social promotion and initiating new summer school 
programs that are designed to give failing students a second chance for 
promotion to the next grade.* The general idea seems very sensible — pro- 
viding additional rime to help all students master the same content is an 
intuitive solution that is grounded in research.^ The summer session, when 
students typically are out of school, seems to be an obvious time to offer 
this extra help. Summer school also makes sense, based on evidence dating 
back to 1906 showing that students, on average, lose a little more than a 
months worth of skill or knowledge over the summer break.*^ 

Research on the summer achievement slide has also established that the 
break has an especially deleterious impaa on poor childrens reading achieve- 
ment. While middle-class childrens test scores essentially plateau during 
the summer months, poor childrens scores show marked declines. As a 
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result of this summer learning difference, the reading 
skills of poor students fall farther behind the skill lev- 
els of their middle-class peers. This difference may rep- 
resent as much as one- third of the typical amount of 
learning that takes place during the regular nine-month 
school year."^ Perhaps even more disconcerting, a long- 
term study led by Karl Alexander and Doris Enrwisle 
established that these income-based summer learning 
differences accumulate over the elementary school years, 
such that poor childrens achievement scores fall far- 
ther and farther behind the scores of their more ad- 
vantaged peers as they progress through school.^ 

Because this research also shows that children of all 
income levels tend to gain at similar rates during the 
nine-month school year, the widening of the test-score 
gap between disadvantaged and advantaged children 
is almost entirely explained by summer learning dif- 
ferences. As a result of this research and the research 
of a few others such as Barbara Heyns, it has become a 
fairly well-established fact that what happens during 
the summer has tremendous implications for under- 
standing and addressing the persistent achievement 
gaps that separate poor and minority children from 
middle-class majority children in the United States.^ 
These findings have tremendous implications for CPS 
in that 86 percent of its students come from low-in- 
come households, and the majority are from racial/ 
ethnic minority backgrounds. 

Summer school, therefore, seems like a good idea 
for a variety of reasons, but how has research weighed 
in on the issue? The most complete and recent source 
of information is the comprehensive quantitative re- 
view, or meta-analysis, by Cooper, Charlton, Valen- 
tine, and Muhlenbruck." These authors concluded that 
the average student enrolled in remedial summer pro- 
grams, like Chicago’s Summer Bridge, would outper- 
form about 56 to 60 percent of similar students not 
enrolled in a summer program. Though there were 
considerable differences across the programs they re- 
viewed, this outcome suggests that summer school may 
be generally regarded as an effective educational inter- 
vention. Such effects prevent the summer slide, help 



students catch up to their peers, and help close the 
achievement gap. 

Across years, subjects, and grades, the Summer 
Bridge adjusted test-score gains (see Figures 2-4a and 
2-4b on page 35) suggest that the students participat- 
ing in the program outperformed between 56 and 73 
percent of similar non-participating students, with the 
clear majority of these gains falling above the high- 
end 60 percent mark noted by Cooper and colleagues. 
Therefore, if summer school programs are, in general, 
effective interventions, and Summer Bridge effects, in 
general, exceed those typically found in other reme- 
dial programs, what does this suggest? Does this find- 
ing malce sense? 1 believe that the answer is “yes” for 
the following reasons. 

In this report, Melissa Roderick, Mimi Engel, Jenny 
Nagaoka, and their colleagues use state-of-the-art quan- 
titative methods to measure the effects of Summer 
Bridge. They have also collected other key data to help 
describe the characteristics of the program, understand 
what it looks like in action, and gain the perspectives 
of participating students and teachers. The authors’ 
analyses address and correct for statistical artifacts, such 
as regression to the mean, that may provide inflated 
estimates of Summer Bridge effects. They also estab- 
lish that the programs effects did not deteriorate in 
the ensuing years following students’ participation. 
They do virtually everything possible to produce un- 
biased appraisals of the efficacy of the program and 
provide convincing evidence that its effects are real 
and impressive. 

Nevertheless, a research purist still may say that 
without a true randomized experiment, we will never 
know how Summer Bridge students would have per- 
formed without the benefit of the program. Yes, an 
experiment like the Tennessee class-size reduction study 
may provide even more convincing evidence of the 
program’s efficacy.^ This would involve randomly de- 
nying summer school services to some students, who 
would serve as a control group, while randomly as- 
signing other students to enroll in the program. There 
may be other alternatives, such as assigning either Sum- 
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mer Bridge or an alternative school year intervention 
at random. l*he achievement gains of those assigned 
to receive Summer Bridge could then be compared to 
the summer gains made by students assigned to re- 
ceive extra help during the ensuing school year. In this 
way, no students would be denied services. The po- 
tential legal and ethical issues involved with such fun- 
damental changes in policy are ones that only CPS, 
and not the Consortium, could address. 

A cynic also may say that, despite the impressive 
gains, only about half of Summer Bridge students meet 
the promotional test-score cutoflFs. Practically speak- 
ing, though, students of different social backgrounds 
and of varying achievement levels all benefit from the 
program and, in general, these benefits are substan- 
tial. Students far from the cutoff score, though, have 
much farther to go and, as a result, many do not meet 
the promotional criterion. Although this is not neces- 
sarily the fault of the program, it suggests that the poli- 
cies for these students may require some rethinking. It 
may not be reasonable to expect these students to meet 
the cutoff score under current conditions. Should an 
alternate program model be available for those stu- 
dents particularly at risk, might more individualized 
instruction, and possibly one-on-one tutoring, help 
them meet the standards? Alternatively, as the greater 
flexibility in the 2000 promotional policy suggests, do 
the criteria for promotion need to be reconsidered? 

The Summer Bridge program has enjoyed success 
because it contains many of the elements that research 
by Cooper et al. and others, suggest distinguish effec- 
tive summer programs: 

• small class sizes; 

• individualized attention to students’ learning 
needs; 

• a careful and highly specific design that im- 
proves the fidelity of the models implemen- 
tation; 

• high attendance rates; 

• substantial academic components aimed at 
teaching reading and math; and 

• integrating summer school experiences with 



O 




those that occur during the regular school 
year, including interactions during Summer 
Bridge between teachers and students who 
were in classrooms together during the regu- 
lar school year.'^ 

At the same time, Summer Bridge seems to avoid 
the pitfalls, noted by Ascher and Austin, Roger, 
and Walbesser of failed and less effective sum- 
mer programs: 

• short program duration; 

• loose organization and little time for advanced 
planning; 

• low academic expectations; 

• discontinuity between the summer curricu- 
lum and the regular school year curriculum; 

• teacher fatigue; and 

• a limited academic focus. 

It is my hope that we can continue to learn from the 
Consortiums research on a program that is, in many 
ways, a model worthy of replication across the coun- 
try. With the Consortiums future research and with 
the school systems ongoing efforts to end social pro- 
motion, Chicago will continue to provide a national 
exemplar of what high-quality mandatory summer 
school programs can achieve. 
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The Chicago Public Schools’ Summer Bridge program 
represents an effort to bring an end to social promO' 
tion through the use of summer school to improve 
student performance on standardized measures of read- 
ing and mathematics achievement. When it began, it 
received considerable public attention. As such, efforts 
to determine whether its impact deser\'ed both the time 
and money provided for it are also likely to generate 
considerable public attention. 

Previous Evaluations of Summer School 

•In our review of the research on the effects of summer 
school, we found that it serves multiple purposes for 
students, families, and communities.^ For example, 
parents and communities hope that, in addition to 
academic instruction, summer school will provide 
positive environments for students and thereby dimin- 
ish juvenile crime. The current need for summer pro- 
grams is driven by changes in American families as 
well as by calls for an educational system that embod- 
ies higher academic standards. 

We examined and integrated the results of 93 evalu- 
ations of summer school. The synthesis revealed that 
summer programs have a clear positive impact on the 
knowledge and skills of participants. The average stu- 
dent who went to summer school jumped over about 
5 to 10 percent of similar students who did not at- 
tend, as measured by achievement test scores. Although 
all students benefited from summer school, students 
from middle-class homes showed larger positive effects 
than students from disadvantaged homes. We suspect 
this is because disadvantaged children often have mul- 
tiple impediments to learning. Even with these im- 
pediments, however, summer school proved effective 
for children from poor families. 



Students at all grade levels benefited from remedial 
sumnier school but students in the earliest grades and 
in high school appeared to benefit most. Consistent 
with our summer learning loss findings, remedial pro- 
grams may have more positive effects on mathematics 
than on reading, though again, the effect on reading is 
clearly positive as well.“ 

Based on these and other results, we recommended 
that summer programs be provided with a stable source 
of funds and that funds be set aside to foster par- 
ticipation, especially among disadvantaged youth. 
The benefits of summer school for achievement are 
clear, and its positive effects may extend beyond the 
schoolyard gates. 

Research Design and Strength of Inferences 

Summer school evaluations vary greatly in terms of 
their quality. Quality is often described as the extent 
to which processes and variables other than the pro- 
gram itself can be ruled out as alternate explanations 
for an observed effect. As a whole, the evaluation of 
the Summer Bridge program is one of the strongest 
single research pieces available on the effects of sum- 
mer school on student achievement. 

The major impediment to strong inferences from 
this particular evaluation arises from its research de- 
sign, or more specifically, how the evaluators con- 
structed the groups of students attending and not 
attending summer school. The research designs em- 
ployed were not the most desirable because students 
were not randomly assigned to the two summer school 
conditions. Rather, summer school attendees were 
selected based on school district policy. Thus, the 
evaluators were left to their own devices to construct 
comparison groups of non-attendees who were as 
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similar as possible to the attendees. This left the evalu- 
ators with less-than-optimal choices regarding how to 
constitute a comparison group. The chosen designs — 
including pre- and post-comparisons for participating 
saidents, cohort analyses, and regression discontinuity 
designs — represent die best available alternatives. 

T he researchers also employed sophisticated statis- 
tical designs and collected a rich database lor answer- 
ing questions. Data were collected using multiple 
methods including standardized achievement tests, sur- 
veys, interviews, and classroom observation. The re- 
sulting evaluation is well written and non-technical, 
with good use of footnotes, tables, and sidebars to ex- 
plain and display necessary technical information. 

Positive Characteristics of the 
Summer Bridge Program 

In our review of summer school research, we also made 
numerous recommendations for summer school 
implementers, which were meant to ensure that pro- 
grams were delivered in the most effective manner 
possible. Several of the defining characteristics of the 
Summer Bridge program are consistent with what we 
found most closely associated with effective summer 
school programs in general. These include small class 
sizes, the alignment of the curriculum and the mea- 
sures used to assess program effectiveness, strong pre- 
program planning, and monitoring of program 
implementation. 

Having these features in the Summer Bridge pro- 
gram design is especially important because past re- 
search on the effectiveness of summer school also 
suggested that large summer programs in a school dis- 
trict serving primarily disadvantaged children would 
be the least likely to succeed. The inclusion of these 
features in the Chicago program likely provides a par- 
tial explanation for its success. 

This should not be taken to mean that the results 
of the Chicago evaluation are free from the influence 
of factors external to the program itself, however. For 
example, one critical feature of the Summer Bridge 



program is the alignment between the curriculum and 
the measures used to assess program effectiveness. With 
respect to high-stakes testing, one concern is that teach- 
ers will “teach the test,’’ That is, students will learn 
how to perform better on the relevant standardized 
test without learning skills that translate into better 
performance in other contexts. Perhaps positively, one 
finding from observations of classrooms revealed that 
teachers spent litde time coaching students on how to 
talce the test. This finding might have been due to the 
highly scripted nature of the curriculum or it might 
have been a function of when the classroom observa- 
tions were made. Coaching might have occurred dur- 
ing the final week of class, just prior to testing but 
after observations were completed (although the find- 
ing that teachers adhered very closely to the scripted 
curriculum lends hope that little coaching occurred 
even in the final fey^ days of school). 

Also, based on student and teacher survey data, the 
Summer Bridge program might influence student 
motivation when taking the high-stakes test. That is, 
it seems likely that some students may not exert full 
effort on the spring administration of the test, which 
may result in a test score that is lower than their abil- 
ity would otherwise indicate. Thus, at least part of the 
apparent impact of the program might not be due to 
instruction per se, but rather to an increase in student 
motivation. If true, this suggests the possibility that 
interventions aimed at improving student test moti- 
vation during the regular school year may be worth 
examining as a cost-effective way of decreasing the 
number of students attending summer school. 

Conclusion 

In general, we found this evaluation to be conducted 
well, especially in light of the constraints faced by the 
evaluators. With a less-than-optimal research design, 
the evaluators took great care to be thorough and to 
ihsure that arguments that both favored and opposed 
the summer program found voice. 
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This evaluation of Chicago’s Summer Bridge program 
is an important contribution to our understanding of 
summer school initiatives. It combines a sophisticated 
analysis of the program with a clear discussion of key 
policy issues. The authors provided valuable insights 
and lessons for educators and policymakers in Boston 
and other cities as they develop their own summer 
programs to raise academic achievement and end so- 
cial promotion. 

In commenting on this evaluation, the Boston sum- 
mer school experience provides a number of interest- 
ing points for comparison. Along with colleagues from 
Northeastern University and Harvard University, I am 
engaged in an evaluation of the Transition Services 
program of the Boston Public Schools. This program 
provides additional literacy and math instruction for 
academically at-risk students in grades three, six, and 
nine, as well as instruction during the summers before 
and after those grades. In the summer of 2000, Boston’s 
Transition summer school program included approxi- 
mately 7,300 students in grades two and three and 
five through nine for 80 hours of instruction (five weeks 
of instruction, four hours per day, four days per week). 
In the summer of 2001, enrollment increased to 8,260 
students in the same grades for the same duration. 

Although similar in their goals, the Chicago and 
Boston programs have several important differences 
that highlight key issues relevant to summer programs. 
One is the policy for excluding particular student popu- 
lation groups. Chicago, for example, excludes students 
with special needs and certain bilingual students, 
whereas Boston includes these populations. In sum- 
mer 2001, students in special and bilingual education 
represented 42 percent of those required to attend 
Bostons summer program. Not surprisingly, this policy 
has important implications — teachers in the Boston 
program, most of whom lacked special training in these 



areas, often commented on the difficult challenges of 
teaching special needs and bilingual students. 

Another key issue on which Boston and Chicago 
differ is the nature of the summer curriculum. Chi- 
cago operates with a relatively prescribed curriculum 
structured around the Iowa Tests of Basic Skills, while 
the curriculum in Boston is more flexible. In fact, a 
major critique by teachers in Bostons summer 2000 
program was the lack of curriculum guidance and 
materials. In summer 2001, this was addressed with a 
more specific set of learning objectives in mathemat- 
ics and English language arts, as well as a set of cur- 
riculum materials for each subject. 

It is interesting to note, however, that teachers in 
both systems had somewhat similar concerns regard- 
ing curriculum, even though the degree of prescrip- 
tion varied. For example, one common critique made 
by teachers in Chicago was the frequent mismatch be- 
tween curriculum materials and the learning needs of 
students with particularly low skills. This same cri- 
tique was made by a significant number of teachers in 
Boston. Also, teachers in both cities raised a concern 
over the rapid pace recommended by the central of- 
fice to cover the curriculum. A lesson to be drawn from 
these comments is the importance of providing a cur- 
riculum that is flexible enough to meet the diverse 
learning needs of a summer school population. Find- 
ing a balance between centralized curriculum guid- 
ance and flexible adaptation is an important 
challenge that tests the skills of curriculum plan- 
ners and teachers. 

Academic achievement is the primary goal of Sum- 
mer Bridge and similar programs. The analysis in this 
report includes a very useful and methodologically in- 
teresting discussion of test-score gains. The use of a 
growth model to chart a trajectory of student test 
scores, the ‘adjusted Summer Bridge gain,” allows the 
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authors to give a more realistic appraisal of gains. In 
addition, sorting students by dieir distance from the 
passing score provides a more nuanced look at the chal- 
lenge faced by the Chicago program. The sorting by 
"level of risk” highlights the significant number of stu- 
dents who are considerably below grade level. For these 
students, distinguishing between achievement scores 
and promotion as outcome measures, as the research- 
ers do in their analysis, is important. Although they 
may not reach the promotional cutoff, their achieve- 
ment gains as measured by test scores are important. 
It is this group of students that teachers are particu- 
larly concerned about in their critique that the cur- 
riculum does not meet the needs of low-skill students. 

A common challenge for summer programs for aca- 
demically at-risk students is low student attendance. 
On this measure, Chicago’s Summer Bridge appears 
to have been quite successful; researchers report atten- 
dance levels in the 90 percent range. By way of con- 
trast, in Bostons summer programs in 2000 and 2001, 
attendance among those students required to attend 
summer school was 66 percent and 67 percent, re- 
spectively. Attendance was particularly problematic in 
the upper grades. In grade nine, for example, only 50 
percent of students told to attend summer school did 
so. Based on my research in several odier Massachusetts 
cities, Boston’s experience in this area is fairly typical. 
Chicago’s success prompts further interest in policies 
and practices that might help to explain this relatively 
high level of attendance. 

The classroom setting and teacher practices received 
extensive attention in this report. In these areas, the 
authors make an important contribution to our un- 
derstanding of successful summer programs. The sup- 
port for small class size among teachers in Chicago is 
not unexpected, but it bears repeating as an impor- 
tant ingredient of a summer program. In Boston as 
well, where class size averages were 10 and 14 in the 
two years of the program, surveys of teachers found 
that small class size was the program’s most frequently 
cited strength. 



Connecting teacher survey responses to student 
achievement is an important exercise in identifying the 
role of teacher practice. Unfortunately, teacher ano- 
nymity in the survey responses complicated this analy- 
sis, requiring the Summer Bridge research team to 
average responses within a grade if there were mul- 
tiple classes in that grade. Wliile recognizing this limi- 
tation, this study reveals a positive association between 
adapting curriculum to student needs and student aca- 
demic achievement. Again, this raises the importance 
of preserving some flexibility in the curriculum to 
allow teachers to tailor instruction to students’ learn- 
ing needs. 

This analysis also shows a positive association be- 
tween teachers’ prior knowledge of students and stu- 
dent academic achievement. This is interesting in light 
of the debate between those advocating summer in- 
struction by teachers who already know the needs of 
individual students and those advocating new teach- 
ers who do not bring preconceived biases and preju- 
dices to their relationships with students. On this 
question, the Boston experience generally supports 
findings for Summer Bridge. In surveys and interviews, 
Boston teachers were much more likely to take the 
position that previous experience with a student was 
beneficial to the learning process. 

The researchers’ use of classroom observations also 
provides an interesting perspective on student achieve- 
ment. The rating scale used for classroom observations 
and the different categories assigned to instruction — 
“tailored” to “insufficient” — are useful strategies for 
capturing the basic nature of classroom practices. Dif- 
ferences observed in instructional practices revealed 
important variation in the classroom experience, even 
when there was a prescribed curriculum. Furthermore, 
this variation is important, as revealed in its correla- 
tion with adjusted learning gains. Consistent with the 
comment in the previous paragraph, the highest learn- 
ing gains were associated with “tailored instruction” 
that included greater adaptation of the curriculum to 
students’ learning needs. 
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In Chapter 6, the authors turn to the central question 
of whether Summer Bridge learning gains are sustained 
over time by developing a quite impressive, compari- 
son group strategy that focused on the differences in 
gains among students above and below the promo- 
tional cutoff compared to the achievement differences 
of comparable groups prior to Summer Bridge, Al- 
though the authors find some sustainability over two 
years, it is modest and limited, particularly for stu- 
dents with the lowest skill levels. The general conclu- 
sion that Summer Bridge provided primarily a 
“one-time boost” for low-achieving students is signifi- 
cant and points to the importance of a long-term strat- 
egy to raise academic achievement. Bostons Transition 
program reflects this thinking in its design as a 15- 
month strategy — two summers and the academic year 
in-between — to raise student achievement. It may be 
that an even longer time frame is needed to make a 
significant impact on student learning. 

In addition, the Interpretive Summary is a very use- 
ful compilation and integration of evaluation findings 
and policy implications. This chapter highlights key 



questions and issues that face administrators and poli- 
cymakers as they design summer programs: staff de- 
velopment, curriculum design, etc. In doing so, a final 
dimension is added to the evaluation that takes it clearly 
beyond Chicago and provides a useful guide for oth- 
ers as they grapple with questions of summer school 
policy and program development to address the needs 
of low-achieving students. 
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1 offer comments from three perspectives: (1) fair and 
effective accountability systems, (2) adequacy in school 
finance, and (3) districts and schools acting jointly as 
a “learning organization” involved in continuous edu- 
cational improvement. 

The Policy, Summer Bridge, the Report, 
and Standards for Accountability Systems 

A recent CRESST/CPRE policy report distills and 
summarizes standards for fair and effective account- 
ability systems (the “Standards”). Interestingly, both 
the Summer Bridge program and Ending Summer 
Bridge: Results from Summer Bridge significandy ad- 
vance the extent to which the underlying accountabil- 
ity policy (grade promotion based on the Iowa Tests 
of Basic Skills, or ITBS) meets the Standards.* In other 
words, both the program — Summer Bridge — and the 
report evaluating the program deserve to be consid- 
ered as important parts of the accountability system 
of the Chicago Public Schools (CPS), the same sort of 
conclusion reached here about adequacy and continu- 
ous educational improvement. 

Although inclusion of reports as part of account- 
ability systems might seem strange, it is unavoidable 
in light of the penultimate Standard on Evaluation 
(number 21), which recommends that “longitudinal 
studies should be planned, implemented, and reported 
evaluating effects of the accountability program” and 
comments, “The primary purpose of educational ac- 
countability systems is to improve instruction and stu- 
dent learning.” The Consortium report, a longitudinal 
evaluation of effects on learning, shows that Summer 
Bridge produced substantial gains in student learning 
in a relatively short period of time (gains of two to six 
months as measured in grade equivalents, depending 
on grade and subject matter, in 90 to 140 hours of 
instruction — up to three times the learning rate of the 



same students in the regular school year). The validity 
of Summer Bridge gains is supported by the findings 
in Chapter 6 on sustainability — on average, students 
maintained summer gains in subsequent school years 
(validation from the same test under different educa- 
tional circumstances rather than from a different mea- 
sure of achievement, as the Standards recommend). 
In documenting these gains, the authors satisfy Stan- 
dard 6, which endorses longitudinal analysis as a means 
of avoiding erroneous inferences about the progress of 
students and schools. 

Standard 10 recommends that “if tests are to help 
improve system performance,” which the Standards 
declare the primary purpose of accountability systems, 
“there should be information provided to document 
that test results are modifiable by quality instruction 
and student effort.” Data provided by the report on 
this point also help meet Standard 2 recommending 
“different types of data from multiple sources,” includ- 
ing “the degree to which students are provided with 
adequate opportunity to learn the content specified in 
content standards and curriculum materials.” Several 
important findings presented in the report deal with 
the effect of instruction — that student attendance and 
satisfaction exceeded the regular school year and that 
student learning is substantially improved by good 
teaching and high-quality schools. 

Regarding instruction, careful classroom observa- 
tions were coded into four categories of the quality of 
instruction: tailored (20 percent of classes), sufficient 
(40 to 50 percent), minimal (10 percent), and insuffi- 
cient (23 percent). Students in the insufficient classes 
gained over two months, on average, less than students 
in tailored classes. (The report convincingly explains, 
why, given problems with the data, these differences 
are almost certainly understated). Although good 
teachers may be born and not made, the characteris- 
tics of the best and worst instruction do not appear 
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mysterious. For example, in tailored instruction, “the 
teachers followed the daily lesson, provided clear and 
accurate instruction about the instructional focus and 
had meaningful and constructive interactions with 
their students,” while with insufficient instruction “the 
lesson was virtually not implemented” in the sense that 
“little or no time was devoted to instruction, there 
we re... extensive periods of down time, and teachers 
provided little structure and inadequate classroom 
management.” Nevertheless, students' high average 
gains probably were due to the close alignment of the 
instructional materials with the test and the high de- 
gree of teacher compliance with the pace of the daily 
lesson plans (90 percent) despite their reservations 
about how well the curriculum met the needs of indi- 
vidual students. Extra help for students and assign- 
ments outside of class also paid offin higher test scores. 

Standard 1 5 on stakes in accountability systems pro- 
vides that “stakes for accountability systems should 
apply to adults and students and should be coordi- 
nated to support system goals” (the comment adds that 
asymmetry or conflict in stakes, consequences for ei- 
ther adults or students but not both, can have unde- 
sirable consequences). The CPS accountability system 
does have consequences for failing schools. At the same 
time, Summer Bridge itself also seems to be away that 
the system, its teachers, and its schools take responsi- 
bility for students at risk of failure. In terms of pro- 
tecting those most at risk, the report has a dual message, 
with data relevant to Standard 20, which requires pub- 
lic reporting of the performance of sub-groups. On 
the one hand, an outstanding accomplishment of Sum- 
mer Bridge was that students of all sub-groups made 
substantia] gains. On the other, although 86 percent 
of the moderate- and low- risk students (students less 
than 1.5 grade equivalents behind at the end of the 
regular school year) were promoted at the end of the 
summer, relatively few — 10 to 33 percent — high-risk 
students (students more than 1.5 grade equivalents 
behind) were promoted. In other words, high-risk stu- 
dents received some benefit but not enough to be pro- 
moted. In this sense. Summer Bridge was not 



structured to take sufficient responsibility and the sys- 
tem remained asymmetrical. 

This asymmetry was technically cured in the year 
2000 by the adoption, in response to civil rights com- 
plaints, of multiple measures for promotional decisions 
at the end of the summer (including teacher and prin- 
cipal recommendations, grades, attendance in sum- 
mer school, and a range of scores on the ITBS). 
Multiple measures also brought the system into com- 
pliance with Standard 7, which states bluntly, “Deci- 
sions about individual students should not be made 
on the basis of a single test.” The result was that, in 
2000 in third grade, 77 percent of students were pro- 
moted, only half of whom made the ITBS cutoff. The 
report makes this comment at the end of Chapter 2: 

Finally, this chapter highlights the central tension of 
creating effective policies under high-stakes testing. 

On the one hand, findings in this chapter suggest 
that using single rest- score cutoffs for promotional 
decisions may set up for fiiilure those students with 
the lowest skills. Despite test-score gains in Summer 
Bridge that were larger than average for these stu- 
dents, very few high-risk students were able to raise 
their test scores to the promotional cutoffs. The ex- 
pansion of the promotional criteria means that stu- 
dents with the lowest skills had a significantly higher 
likelihood of being promoted to the next grade in 
2000, which could be. interpreted as a means of re- 
warding work effort and an acknowledgement of 
learning gains despite not having met the cutoffs. 

On the other hand, one could argue that the system 
is no longer working toward ending social promo- 
tion and that many students are now being promoted 
without adequate skills. 

Confirming the presence of a genuine tension, the 
system's broadening of the promotional standards also 
introduced a different kind of asymmetry by lowering 
the incentives for student work, a risk made more plau- 
sible by a different study cited in the same chapter 
showing that test scores of certain sixth- and eighth- 
grade students who did not need to pass the test actu- 
ally declined over the summer. A logical question is 
whether providing more of the most effective re- 
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sources to the students most at risk could have raised 
their promotion rate to an acceptable level, preserv- 
ing the incentives for student work, while avoiding 
setting up high-risk students for failure. That idea 
is a good transition to the next topic — adequacy in 
educational finance. 

Summer Bridge and Adequacy in School Finance 

The authors explicitly offer the explanation that addi- 
tional resources were responsible for gains in student 
learning: extra instruction over and above the school 
year, smaller class sizes than the school year (16 com- 
pared with 29-31), a curriculum carefully aligned with 
the test, and fully subsidized curriculum materials and 
lesson plans. These substantial resources targeted on 
students at risk plausibly provided the significant op- 
portunity to learn that was responsible for test- score 
gains . The strong incentives produced another re- 
source — high student effort. Research cited by the 
authors shows that students learn more with personal 
support (personalism) and strong expectations for stu- 
dents to work hard (academic press). In the report, we 
learn that over 80 percent of sixth and eighth graders 
reported that they worked harder in summer school 
than the regular year. Over 85 percent reported that 
teachers had more time to help them in Summer Bridge 
than during the regular school year — a magnifier of 
incremental instruction. 

Differential resources within Summer Bridge also 
had an impact. In addition to the quality of instruc- 
tion discussed above, other resources that had an im- 
pact on test-score gains were school quality, teachers 
knowing their students prior to the program, and a 
class size of 12 or fewer students. Regarding school 
quality, even after accounting for the fact that low- 
achieving schools tended to serve lower- achieving stu- 
dents, adjusted learning gains varied significandy across 
schools. Students who attended high-performing 
schools (those with, higher school year ITBS scores) 
had larger gains. Adjusted learning gains in reading 
were significantly greater in all grades in schools where 
teachers reported knowing a larger proponion of their 



students before Summer Bridge. This large effect oc- 
curred even after controlling for the demographic and 
achievement characteristics of students and schools. 
Regarding class size, the report showed that having a 
class size of 12 or fewer, and knowing the teacher, made 
a substantial difference in measures of both academic 
press and personalism. 

Such findings make Summer Bridge a close fit with 
work I have done on adequacy in school finance." In 
that work, I suggest that educational resources are ad- 
equate to the extent that they allow low-achieving stu- 
dents to reach proficiency on standards applicable to 
all students in a state. Resources should be targeted in 
two senses: on low-achieving students and on inter- 
ventions with the demonstrated capacity to achieve 
the goal. In one sense, there would be no upper limit 
to the resources: proficiency costs what it costs. But 
there also would be no lower limit in the sense that 
low-cost programs achieving the goal could (and prob- 
ably would) be preferred. Because the entidement to 
extra resources is contingent on their effectiveness, 
adequacy necessarily has an experimental aspect. Judi- 
cial, legislative, and educational remedies would be 
implemented, evaluated, and adjusted according to 
data on impact. 

Based on the authors’ findings. Summer Bridge rep- 
resents a solid but partial success story of educational 
adequacy. Substantial, extra, and targeted resources 
were provided to low-achieving students. Many did 
meet the promotional cutoff, a working definition of 
adequacy. The programs success was partial for the 
same reason it suffered against standards of account- 
ability: less than half the students at greatest risk made 
the cutoff and the program was compromised in away 
that may have affected its incentive power. Rather than 
watering down the cutoff, the district might have ex- 
plored more and different kinds of targeting on the 
students most at risk: allocating them to the best 
schools and teachers, allocating them to good teachers 
who knew them from the school year, improving pro- 
fessional development, and financing the most effec- 
tive kinds of instruction (for example, extra individual 
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attention), and so forth. Based on the substantial gains 
already demonstrated, full adequacy seemed well 
within reach, and the civil rights complaint might have 
been better directed at extra resources than lower stan- 
dards (though, unfortunately, less supported legally in 
the state of Illinois). 

Of course, to say ‘‘lower standards'' implies that we 
know the answer to the question posed by the report — 
whether the new, multiple measures for promotion are 
a more valid indicator of adequate skills (as the Stan- 
dards assume) or merely a convenient way to promote 
more students. The Standards seem ambiguous on this 
point. In the Standards, a single test score is presumed 
invalid, and multiple measures are presumed more 
valid. At the same time, multiple measures are only 
more valid to the extent that they are correlated with 
adequate skills, a fact, which to my knowledge, has 
not been demonstrated. A watered down system would 
not be as effective at improving teaching and learning, 
the primary purpose of accountability systems. Quite 
some time ago, a South Carolina legislator explained 
to me that minimum competency tests (the anti-so- 
cial promotion tool of that day) were a ‘"tightrope” for 
low-achieving minority students that created the risk 
of increased failure and stigma but avoided the risk of 
inferior education and “being ignored” by the system. 
The tightrope image seems a good one because it is 
possible to fall off in either direction. 

Such gaps in our knowledge — about the effects of 
further targeting and the validity of multiple mea- 
sures — are a good transition to the next topic — con- 
tinuous improvement and organizational learning. 

Continuous Improvement and 
Organizational Learning 

Chapter 4 of the report makes the following rec- 
ommendation regarding teacher training for Sum- 
mer Bridge: 

An important policy question is whether such ac- 
tivities could be further encouraged, both in the Sum- 
mer Bridge program and in summer programs in 
general. Summer Bridge teachers seem to have been 



highly sensitive to the messages they received in train- 
ing about adapting the curriculum. For example, in 
interviews, teachers reported spending more time on 
reading than mathematics, a strategy that had been 
recommended in training sessions. Summer pro- 
grams can send the me.ssage to teachers that meeting 
individual student needs is as important as covering 
the required content, and could provide teachers extra 
supports and training in this area. 

This is an example of the transition from evaluation 
to continuous improvement by which the district, 
schools, and teachers could function jointly as a learn- 
ing organization. If, as the Standards say, “The pri- 
mar>^ purpose of educational accountability systems is 
to improve instruction and student learning,” educa- 
tional improvement is part of accountability. Begin- 
ning with a finding (like meeting the needs of 
individual students) and building a program of sup- 
port and training would create a learning loop. This is 
similar to what is described in an Annenberg report as 
an “inquiry cycle” with six key stages, all of them nec- 
essary and iterative but occurring in no defined order 
in effective organizations: establish desired outcomes, 
define the questions, collect and organize data, make 
meaning of the data, take action, and assess and evalu- 
ate actions.^ 

Getting beyond accountability in the narrow sense 
is not easy. In many ways, the use of data for school 
improvement requires a transition from functioning 
as a bureaucratic organization to functioning as a 
“learning organization.” Scribner, Cockrell, Cockrell, 
and Valentine make this comment: “Organizational 
learning. ..has been posited as a process that can lead 
to second-order change, that is, change that is the re- 
sult of the critical evaluation of underlying values and 
assumptions that guide behavior.”'^ Resnick and 
Glenn an describe the task as creating “nested learning 
communities” involving six principles: a commitment 
to an effort-based concept of intelligence and educa- 
tion; a focus on classroom instruction throughout the 
district; a culture emphasizing learning and two-way 
accountability — the core elements of nested learning 
communities — throughout the system; continuing 
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professional development for all staff that is based in 
schools and linked to the instructional program; and 
coherence in standards, curriculum, assessment, and 
professional development.^ (T hese are the guiding 
principles of the Institute for Learning, an organiza- 
tion that collaborates with a number of urban districts). 

I find it helpful to distinguish the distinct roles of 
districts and schools in this process. Districts can pro- 
vide incentives, leadership, organization, training, and 
targeted support. Schools provide instruction under 
the leadership of their principals and staffs Function- 
ing as a learning organization involves close coordina- 
tion and continuous improvement of the two roles. 
As described in the report, the Summer Bridge pro- 
gram already has many of the elements of a nested 
learning community on a programmatic rather than 
systemwide basis — an opportunity for students to learn 
through extra effort; a program built around classroom 
instruction (the ITBS curriculum and materials); con- 
tinuing professional development; and coherence in 
standards, assessment, and professional development. 
A useful next .step toward second-order change would 
be allocating more strategic resources to the vision that 
all children can reach proficiency and evaluating the 
impact of these resources on the chosen goal. 

Conclusion 

The three perspectives of accountability, adequacy in 
school finance, and continuous improvement have a 
common emphasis on improving student achievement. 
Accountability can be thought of narrowly as creating 
incentives and observing outcomes, as in ending so- 
cial promotion and seeing if scores improve. Improve- 
ments from incentives alone are rarely sufficient to 
reach the goal, however, which opens the door to the 
broader idea of accountability as information about 



how to improve. To its credit, this report identifies 
what works, how much, and what else might be done 
to reach the goal of proficiency for all students. 



’ Policy Brief 5 of the National Center for Research on Evalu- 
ation, Standards, and Student Testing (Winter 2002). 

^ WH. Clune, .ed., “Equity and adequacy in education: Issues 
for policy and finance” [special Educational Policy % no. 

4 (1994): 365-375. Seedso WH. Clime, oi Equity and 

Adequacy In Education FinancCy Issues and Perspectives^ edited 
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103, no. 1 (2000): 55-75. 

^ L. Keeney, Using Data for School Improvement: Report on the 
Second Practitioners Conference for Annenherg Challenge Sites, 
Houston, May 1998 (Providence, RI: Annenherg Institute for 
School Reform at Brown University, 1998). 

J.P. Scribner, K.S. Cockrell, D.H. Cockrell, and J.W. Valen- 
tine, “Creating professional communities in schools through 
organizational learning: An evaluation of a school improve- 
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proficiencies: Systemic school reform through high stakes as- 
sessments and a network of schools” (paper prepared for the 
annual meeting of the American Educational Researchers As- 
sociation, New Orleans, LA, April 2002). 
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